RNC ProPack
From MultimediaWiki
| Revision as of 10:27, 9 March 2011 Kostya (Talk | contribs) (RNC description, version 2 from VAG, version 1 from ScummVM) ← Previous diff |
Revision as of 18:56, 9 March 2011 VAG (Talk | contribs) (fix header, add info about encryption, fix broken algo) Next diff → |
||
| Line 1: | Line 1: | ||
| RNC is a general data compression library (like [[zlib]]), name probably stands for Rob Northern Compression (got it from [http://code.google.com/p/corsix-th/wiki/RNC here]). | RNC is a general data compression library (like [[zlib]]), name probably stands for Rob Northern Compression (got it from [http://code.google.com/p/corsix-th/wiki/RNC here]). | ||
| + | |||
| + | Packer DOS binary, along with assembly decoding modules (barely useable nowdays) can be freely downloaded [http://aminet.net/package/util/pack/RNC_ProPack here]. | ||
| There are several known variants in the wild, version 1 was employed to compress data files in various games, like Beneath a Steel Sky or Dungeon Keeper, version 2 seems to be used in [[ETV]] codec. | There are several known variants in the wild, version 1 was employed to compress data files in various games, like Beneath a Steel Sky or Dungeon Keeper, version 2 seems to be used in [[ETV]] codec. | ||
| - | === Header === | + | == Header == |
| All numbers are stored as big-endian. | All numbers are stored as big-endian. | ||
| Line 11: | Line 13: | ||
| 4 bytes unpacked data length | 4 bytes unpacked data length | ||
| 4 bytes packed data length | 4 bytes packed data length | ||
| - | 2 bytes CRC | + | 2 bytes CRC-16 of packed data |
| - | 2 bytes tail CRC | + | 2 bytes CRC-16 of unpacked data |
| - | 2 bytes unknown | + | 2 bytes overlap size (used for inplace unpacking) |
| + | |||
| + | == Compression == | ||
| + | |||
| + | As an extra protection measure against curious hackers compressed data can be encrypted with secret 16-bit key (supplied to decompression routine by programmer.) In that case all byte literals decoded as follows: | ||
| + | |||
| + | byte ^= key; key = _rotr(key, 1); | ||
| === Version 0 === | === Version 0 === | ||
| Line 72: | Line 80: | ||
| for (i = 0; i < length; i++) | for (i = 0; i < length; i++) | ||
| *dst++ = get_byte(); | *dst++ = get_byte(); | ||
| + | continue; | ||
| } | } | ||
| } | } | ||
| Line 109: | Line 118: | ||
| } | } | ||
| offset = (value << 8) + get_byte() + 1; | offset = (value << 8) + get_byte() + 1; | ||
| + | |||
| [[Category:General Compression Methods]] | [[Category:General Compression Methods]] | ||
Revision as of 18:56, 9 March 2011
RNC is a general data compression library (like zlib), name probably stands for Rob Northern Compression (got it from here).
Packer DOS binary, along with assembly decoding modules (barely useable nowdays) can be freely downloaded here.
There are several known variants in the wild, version 1 was employed to compress data files in various games, like Beneath a Steel Sky or Dungeon Keeper, version 2 seems to be used in ETV codec.
Contents |
Header
All numbers are stored as big-endian.
3 bytes signature "RNC" 1 byte version 4 bytes unpacked data length 4 bytes packed data length 2 bytes CRC-16 of packed data 2 bytes CRC-16 of unpacked data 2 bytes overlap size (used for inplace unpacking)
Compression
As an extra protection measure against curious hackers compressed data can be encrypted with secret 16-bit key (supplied to decompression routine by programmer.) In that case all byte literals decoded as follows:
byte ^= key; key = _rotr(key, 1);
Version 0
This is uncompressed data.
Version 1
Decoder source is in ScummVM repository.
Bits are packed into bytes and those are interspersed with byte values (like in KMVC for example).
Data is packed with LZ77 scheme into blocks, each block has three Huffman tables for literal and copy lengths and offsets.
Huffman tables:
5 bits - number of entries (0-16) 4x<number> bits - code lengths
for codes reconstruction and tree reading pleas refer to decoder code.
Block starts with three tables description and 16-bit number of (literal, copy) pairs.
while (!end) {
read literal length table
read offset table
read copy length table
blocks = get_bits(16);
while (blocks--) {
read_length = get_value(literal_table);
while (read_length--)
*dst++ = get_byte();
if (blocks) {
offset = get_value(offset_table);
length = get_value(copy_len_table);
memmove(dst, dst - offset, length);
dst += length;
}
}
}
Version 2
This seems to be designed to be faster on decoding so it does not use Huffman tables (but it's still LZ77 scheme with the same bitreader).
get_bits(2); // unused
while (!end) {
if (!get_bit()) {
*dst++ = get_byte();
} else {
if (!get_bit()) {
length = 4 + get_bit();
if (get_bit()) {
length = (length - 1) * 2 + get_bit();
if (length == 9) {
length = (get_bits(4) + 3) * 4;
for (i = 0; i < length; i++)
*dst++ = get_byte();
continue;
}
}
offset = get_offset();
} else {
if (get_bit()) {
if (get_bit()) {
length = get_byte() + 8;
if (length == 8) {
get_bit();
continue; //dunno why
}
} else {
length = 3;
}
offset = get_offset();
} else {
offset = get_byte() + 1;
}
}
memmove(dst, dst - offset, length);
dst += length;
}
}
Offset is coded like that:
value = 0;
if (get_bit()) {
value = get_bit();
if (get_bit()) {
value = value * 2 + 4 + get_bit();
if (!get_bit())
value = value * 2 + get_bit();
} else if (value == 0)
value = get_bit() + 2;
}
offset = (value << 8) + get_byte() + 1;
