Difference between revisions of "RNC ProPack"

From MultimediaWiki
Jump to: navigation, search
(RNC description, version 2 from VAG, version 1 from ScummVM)
 
(fix header, add info about encryption, fix broken algo)
Line 1: Line 1:
 
RNC is a general data compression library (like [[zlib]]), name probably stands for Rob Northern Compression (got it from [http://code.google.com/p/corsix-th/wiki/RNC here]).
 
RNC is a general data compression library (like [[zlib]]), name probably stands for Rob Northern Compression (got it from [http://code.google.com/p/corsix-th/wiki/RNC here]).
 +
 +
Packer DOS binary, along with assembly decoding modules (barely useable nowdays) can be freely downloaded [http://aminet.net/package/util/pack/RNC_ProPack here].
  
 
There are several known variants in the wild, version 1 was employed to compress data files in various games, like Beneath a Steel Sky or Dungeon Keeper, version 2 seems to be used in [[ETV]] codec.
 
There are several known variants in the wild, version 1 was employed to compress data files in various games, like Beneath a Steel Sky or Dungeon Keeper, version 2 seems to be used in [[ETV]] codec.
  
=== Header ===
+
== Header ==
  
 
All numbers are stored as big-endian.
 
All numbers are stored as big-endian.
Line 11: Line 13:
 
   4 bytes  unpacked data length
 
   4 bytes  unpacked data length
 
   4 bytes  packed data length
 
   4 bytes  packed data length
   2 bytes  CRC
+
   2 bytes  CRC-16 of packed data
   2 bytes  tail CRC
+
   2 bytes  CRC-16 of unpacked data
   2 bytes  unknown
+
   2 bytes  overlap size (used for inplace unpacking)
 +
 
 +
== Compression ==
 +
 
 +
As an extra protection measure against curious hackers compressed data can be encrypted with secret 16-bit key (supplied to decompression routine by programmer.) In that case all byte literals decoded as follows:
 +
 
 +
  byte ^= key; key = _rotr(key, 1);
  
 
=== Version 0 ===
 
=== Version 0 ===
Line 72: Line 80:
 
             for (i = 0; i < length; i++)
 
             for (i = 0; i < length; i++)
 
                 *dst++ = get_byte();
 
                 *dst++ = get_byte();
 +
            continue;
 
           }
 
           }
 
         }
 
         }
Line 109: Line 118:
 
   }
 
   }
 
   offset = (value << 8) + get_byte() + 1;
 
   offset = (value << 8) + get_byte() + 1;
 +
  
 
[[Category:General Compression Methods]]
 
[[Category:General Compression Methods]]

Revision as of 11:56, 9 March 2011

RNC is a general data compression library (like zlib), name probably stands for Rob Northern Compression (got it from here).

Packer DOS binary, along with assembly decoding modules (barely useable nowdays) can be freely downloaded here.

There are several known variants in the wild, version 1 was employed to compress data files in various games, like Beneath a Steel Sky or Dungeon Keeper, version 2 seems to be used in ETV codec.

Header

All numbers are stored as big-endian.

 3 bytes   signature "RNC"
 1 byte    version
 4 bytes   unpacked data length
 4 bytes   packed data length
 2 bytes   CRC-16 of packed data
 2 bytes   CRC-16 of unpacked data
 2 bytes   overlap size (used for inplace unpacking)

Compression

As an extra protection measure against curious hackers compressed data can be encrypted with secret 16-bit key (supplied to decompression routine by programmer.) In that case all byte literals decoded as follows:

 byte ^= key; key = _rotr(key, 1);

Version 0

This is uncompressed data.

Version 1

Decoder source is in ScummVM repository.

Bits are packed into bytes and those are interspersed with byte values (like in KMVC for example).

Data is packed with LZ77 scheme into blocks, each block has three Huffman tables for literal and copy lengths and offsets.

Huffman tables:

 5 bits - number of entries (0-16)
 4x<number> bits - code lengths

for codes reconstruction and tree reading pleas refer to decoder code.

Block starts with three tables description and 16-bit number of (literal, copy) pairs.

 while (!end) {
   read literal length table
   read offset table
   read copy length table
   blocks = get_bits(16);
 
   while (blocks--) {
     read_length = get_value(literal_table);
     while (read_length--)
       *dst++ = get_byte();
     if (blocks) {
       offset = get_value(offset_table);
       length = get_value(copy_len_table);
       memmove(dst, dst - offset, length);
       dst += length;
     }
   }
 }

Version 2

This seems to be designed to be faster on decoding so it does not use Huffman tables (but it's still LZ77 scheme with the same bitreader).

 get_bits(2); // unused
 while (!end) {
   if (!get_bit()) {
     *dst++ = get_byte();
   } else {
     if (!get_bit()) {
       length = 4 + get_bit();
       if (get_bit()) {
         length = (length - 1) * 2 + get_bit();
         if (length == 9) {
           length = (get_bits(4) + 3) * 4;
           for (i = 0; i < length; i++)
               *dst++ = get_byte();
           continue;
         }
       }
       offset = get_offset();
     } else {
       if (get_bit()) {
         if (get_bit()) {
           length = get_byte() + 8;
           if (length == 8) {
              get_bit();
              continue; //dunno why
           }
         } else {
           length = 3;
         }
         offset = get_offset();
       } else {
         offset = get_byte() + 1;
       }
     }
     memmove(dst, dst - offset, length);
     dst += length;
   }
 }

Offset is coded like that:

 value = 0;
 if (get_bit()) {
   value = get_bit();
   if (get_bit()) {
     value = value * 2 + 4 + get_bit();
     if (!get_bit())
       value = value * 2 + get_bit();
   } else if (value == 0)
     value = get_bit() + 2;
 }
 offset = (value << 8) + get_byte() + 1;