Kostya: inter-frame DCT tile description

2026-05-23T09:07:38Z

inter-frame DCT tile description

← Older revision		Revision as of 02:07, 23 May 2026
Line 74:		Line 74:
	}		}
	while (idx < 64) {		while (idx < 64) {
	add_q = quant_sel > 14 ? (quant_sel < idx ? 2 : 8) : (quant_sel <= idx ? 16 : 8);		add_q = quant_sel > 14 ? (quant_sel > idx ? 2 : 8) : (quant_sel <= idx ? 16 : 8);
	if (get_bit()) {		if (get_bit()) {
	coef[idx++] = get_ac6_63_nozero() * add_q;		coef[idx++] = get_ac6_63_nozero() * add_q;
Line 113:		Line 113:

	Skip value codes:		Skip value codes:
	* <code>1</code> -- 1		* <code>1</code> -- end of block
	* <code>0 00</code> -- 3		* <code>0 00</code> -- 3
	* <code>0 01</code> -- 4		* <code>0 01</code> -- 4
Line 132:		Line 132:
	Unlike intra frame, this mode stores tile types and data interleaved, without any run coding and there are only four types (requiring two bits):		Unlike intra frame, this mode stores tile types and data interleaved, without any run coding and there are only four types (requiring two bits):
	* 0 -- skip (leave current block unchanged)		* 0 -- skip (leave current block unchanged)
	* 1 -- DCT update ~~block~~		* 1 -- DCT update tile
	* 2 -- repeat copy operation (i.e. if intra frame tile at the current position copied last/top block, copy it again as well)		* 2 -- repeat copy operation (i.e. if intra frame tile at the current position copied last/top block, copy it again as well)
	* 3 -- re-use cached DCT tile		* 3 -- re-use cached DCT tile

	=== DCT update ~~block~~ ===		=== DCT update tile ===

			This DCT tile uses DCT coefficients data (luma block and DCs from chroma blocks) as its base (without updating the reference).


	~~TODO~~

	=== Cached DCT tile ===		=== Cached DCT tile ===
Line 146:		Line 148:
	For example, if first e.g. tile 24 is selected from the cache, next operation may simply tell to use the next one (tile 25) instead of sending its index explicitly.		For example, if first e.g. tile 24 is selected from the cache, next operation may simply tell to use the next one (tile 25) instead of sending its index explicitly.

			DCT data starts with 6-bit index of last non-zero luma coefficient (in zigzag order).

			Coding mode for luma block depends on reference block quantiser selector.

			If that selector value is less than 15 then coefficients are coded as:
			* when reference coefficient is non-zero and only DC is coded -- as a signed 4-bit difference (with sign inverted if base coefficient is negative);
			* when reference coefficient is non-zero -- as a signed 3-bit difference (with sign inverted if base coefficient is negative);
			* when reference coefficient is zero -- the same way as DC difference in intra DCT tiles.

			Otherwise there's also a different mode for small coefficients: -1/0/1 for zero base coefficients (<code>11</code>, <code>0</code> and <code>10</code> correspondingly) and 0/&mnplus;1 for non-zero ones. The rest is coded as DC differences.

			Chroma DCT blocks use the same small coefficients mode to code DCs (as that's the only coefficient with the reference available) and JPEG-style AC coding (zero-run plus number of bits in coefficient plus coefficient mantissa bits).

	== DCT quantisation tables ==		== DCT quantisation tables ==

	The packet should be 129 bytes long: packet ID and two 64-byte quantisation tables (for luma and for chroma)		The packet should be 129 bytes long: packet ID and two 64-byte quantisation tables (for luma and for chroma).

Kostya: start filling known information

2026-05-22T17:02:35Z

start filling known information

New page

MVS is an extension (new coding methods) of VNC protocol. It operates on 8x8 blocks (or tiles) and employs DCT in YUV colourspace. There is no sub-sampling, so 8x8 tile codes three 8x8 block for each components (though chroma components usually have the reduced number of coded coefficients).

Packet data begins with a byte signalling its type, the rest depends on type:
* 0 - intra frame
* 1 - inter frame
* 2 - DCT quantisation tables

In general, first decoder should receive quantisation tables, then intra frame and then some inter frames (which update data transmitted in previous frames).

Bitstream is stored MSB first in big-endian format.

== Intra frame ==

Packet payload begins with its ID (0) and two bytes with DCT quantiser selectors. Then there is 24-bit big-endian size of tile metadata followed by tile metadata and the rest is tile data. Data should end with 8-bit code for 'm' (not necessarily at the byte boundary though).

Tiles can either be simple fill operations, copy already decoded tile operation, DCT block or re-using some previously decoded DCT block again.

=== Intra tile metadata ===

Tile metadata is stored in run form: there is 3-bit tile type code followed by variable-length code for the number of tiles with that type.

Tile type run codes are:
* <code>0000-1110</code> --- 1-15 tiles
* <code>11110 + 8-bit code</code> --- 16-271 tiles
* <code>111110 + 15-bit code</code> --- 16-32783 tiles
* <code>111111 + 22-bit code</code> --- 16-4194319 tiles

Tile types are:
* 0 --- fully white block
* 1 --- paint previous block again (one to the left from the current one or last one in the previous row for the first block of the row)
* 2 --- paint block from above the current position
* 3 --- black and white block: bit mask from tile data codes which pixels of the block are black (0) or white (1)
* 4 --- fill or two-colour pattern block (see below)
* 5 --- DCT block (see below)
* 6 --- cached DCT tile: get 16-bit index, draw DCT tile from the cache
* 7 --- paint last used DCT tile (either from previously decoded DCT block or last used cached one, whichever is more recent) again

=== Fill/two-colour block ===

This mode either fills tile with single colour or paints a pattern using two colours. First bit in tile data tells which mode is that, then there are 1-2 colours in 8-6-6 YUV format (8 bits per luma, top 6 bits per chroma component, see conversion formulae below) and for 2-colour mode there is fill pattern as well.

=== DCT tile ===

DCT tile comprises data for three blocks, one for each component. For encoding, RGB data is converted to YUV, FDCT is performed on each component, then DC prediction is applied, and finally all block data is coded in a special way using general codes (usually Rice codes) dependent on the coefficient position (i.e. DCs are coded in one way, coefficients 1-6 in zigzag order are coded in a different way and the rest of coefficients are coded in yet another way).

Block format:
chroma_dc_coded = get_bit();
quant_sel = get_bit() ? header_byte2 : header_byte1;
if chroma_dc_coded {
cr_blk[0] = (last_cr_dc / 2 - get_dc()) * 2;
cb_blk[0] = (last_cb_dc / 2 - get_dc()) * 2;
last_cr_dc = cr_blk[0];
last_cb_dc = cb_blk[0];
} else {
cr_blk[0] = last_cr_dc;
cb_blk[0] = last_cb_dc;
}
y_blk[0] = last_y_dc - get_dc();
last_y_dc = y_blk[0];

idx = 1;
while (idx < 6) {
add_q = quant_sel > 14 ? 2 : (quant_sel <= idx ? 16 : 8);
if (get_bit()) {
coef[idx++] = get_ac1_5_nozero() * add_q;
} else {
// decode skip/level value
if (is_skip) {
idx += value;
} else {
coef[idx++] = value * add_q;
}
}
}
while (idx < 64) {
add_q = quant_sel > 14 ? (quant_sel < idx ? 2 : 8) : (quant_sel <= idx ? 16 : 8);
if (get_bit()) {
coef[idx++] = get_ac6_63_nozero() * add_q;
} else {
// decode skip/level value
if (is_skip) {
idx += value;
} else {
coef[idx++] = value * add_q;
}
}
}

For DC prediction initial DC predictors are set to zero.

DC codes: get prefix as unary code of ones (e.g. <code>0</code> = 0, <code>1110</code> = 3), get sign bit, get mantissa bits depending on prefix, negate value if sign bit is set and return it.

Code structure:
* <code>0 s x</code> -- codes ±1
* <code>10 s x</code> -- codes ±2-3
* <code>110 s xx</code> -- codes ±4-7
* <code>11..10 s xxx</code> -- ±((prefix - 2) * 8 + <code>xxx</code>)

Coefficient 1-4 no-zero codes: get prefix as unary code of ones, get sign bit, for prefixes 0-3 code is ±<code>(prefix * 4) + get_bits(2) + 2</code>, for other prefixes it is ±<code>(prefix - 4) * 8 + get_bits(3) + 18</code>.

Coefficient 6-63 no-zero codes are similar to DC codes: get prefix as unary code of ones (e.g. <code>0</code> = 0, <code>1110</code> = 3), get sign bit, get mantissa bits depending on prefix, negate value if sign bit is set and return it.

Code structure:
* <code>0 s x</code> -- codes ±2-3
* <code>10 s xx</code> -- codes ±4-7
* <code>1..10 s xxx</code> -- ±((prefix - 1) * 8 + <code>xxx</code>)

Other coefficient codes use special scheme signalled by 2-bit code type:
* 0 -- new level value is 1
* 1 -- new skip value (see below)
* 2 -- re-use previous non-zero coefficient value
* 3 -- re-use negated previous non-zero coefficient value

Skip value codes:
* <code>1</code> -- 1
* <code>0 00</code> -- 3
* <code>0 01</code> -- 4
* <code>0 10</code> -- 5
* <code>0 11 xxx yyy ...</code> -- 6+n (read 3-bit values and add them to the initial value of 6, stop when 3-bit value is not equal to 7)

=== Cached DCT tile ===

Decoder maintains a circular buffer of 65000 (not 65536!) DCT tiles. Every time a DCT block is decoded, it is put into this buffer and may be re-used later.

== Inter frame ==

In this mode tile may reference previously decoded tile data e.g. re-use copy operation or update previously decoded DCT coefficients.

Like intra frame, packet data starts with ID and two bytes of header, telling the maximum number of Cr and Cb coefficients that may be coded in DCT blocks. Data should end with <code>"mvs\0"</code> sequence (not necessarily at the byte boundary though).

Unlike intra frame, this mode stores tile types and data interleaved, without any run coding and there are only four types (requiring two bits):
* 0 -- skip (leave current block unchanged)
* 1 -- DCT update block
* 2 -- repeat copy operation (i.e. if intra frame tile at the current position copied last/top block, copy it again as well)
* 3 -- re-use cached DCT tile

=== DCT update block ===

**TODO**

=== Cached DCT tile ===

Unlike intra mode, there is a flag telling whether decoder should read full 16-bit tile index or use the next tile in cache.

For example, if first e.g. tile 24 is selected from the cache, next operation may simply tell to use the next one (tile 25) instead of sending its index explicitly.

== DCT quantisation tables ==

The packet should be 129 bytes long: packet ID and two 64-byte quantisation tables (for luma and for chroma)

== YUV colourspace ==

The colourspace seems to use these coefficients:
Y = 0.314520 * R + 0.572543 * G + 0.112937 * B
Cr = 0.527292 * R - 0.440418 * G - 0.086874 * B
Cb = -0.177695 * R - 0.323471 * G + 0.501166 * B

Or, for the reverse operation
R = Y + 1.3 * Cr
G = Y - 0.71414 * Cr - 0.34914 * Cb
B = Y + 1.77 * Cb

[[Category:Networking Protocols]]

MVS - Revision history

Kostya: inter-frame DCT tile description

Kostya: start filling known information