Sorenson Video 3: Difference between revisions

From MultimediaWiki
Jump to navigation Jump to search
No edit summary
(6 intermediate revisions by 4 users not shown)
Line 4: Line 4:


Video codec apparently based on an early H.264 draft.
Video codec apparently based on an early H.264 draft.
The decoder in FFmpeg produces heavy artifacting for some uncommon samples.


== Decoding Process ==
== Decoding Process ==
Line 14: Line 12:


=== Macroblock layer ===
=== Macroblock layer ===
Each macroblock starts with Golomb code signalling MB type:
Each macroblock starts with [[Golomb]] code signalling MB type:
* 0 - skip block
* 0 - skip block
* 1-7 - inter 16x16 blocks, type specifies prediction direction
* 1-7 - inter 16x16 blocks, type specifies prediction direction
* 8 and 33 - intra 4x4 block (type 33 means that there are no coefficients for this block)
* 8 and 33 - intra 4x4 block (type 33 means that there are no coefficients for this block)
* all other values are for inter 4x4 blocks, type is used to set prediction direction and coded block pattern
* 9-32 - intra 16x16 blocks, type is used to set prediction direction and coded block pattern
In any case coefficients are stored in 4x4 (sub)blocks.
 
Dezigzag pattern (from [[H.264]]):
 
  o-->o-->o  o
          |  /|
  o  o  o / o
  | / |  |/  |
  o  o  o  o
    /
  o-->o-->o-->o
 
=== Coefficient decoding ===
Each coefficient is stored as [[Golomb]] codeword, last bit is coefficient sign, code = 0 means end of nonzero coefficients.
 
Codes for block type 3:
{| border="1" cellpadding="5" style="border-collapse: collapse; border-style: dashed; border-color: #2f6fab;"
|- bgcolor="#f0f0f0" |
! code !! run !! value
|-
| 0-2
| 0
| '''code'''
|-
| 3
| 1
| 1
|-
| 4-...
| '''code''' & 0x3
| (('''code''' + 9) >> 2) - '''run'''
|}
 
Codes for intra block types:
{| border="1" cellpadding="5" style="border-collapse: collapse; border-style: dashed; border-color: #2f6fab;"
|- bgcolor="#f0f0f0" |
! code !! run !! value
|-
| 0 || 0 || 0
|-
| 1 || 0 || 1
|-
| 2 || 1 || 1
|-
| 3 || 0 || 2
|-
| 4 || 2 || 1
|-
| 5 || 0 || 3
|-
| 6 || 0 || 4
|-
| 7 || 0 || 5
|-
| 8 || 3 || 1
|-
| 9 || 4 || 1
|-
| 10 || 1 || 2
|-
| 11 || 1 || 3
|-
| 12 || 0 || 6
|-
| 13 || 0 || 7
|-
| 14 || 0 || 8
|-
| 15 || 0 || 9
|-
| 16-...
| '''code''' & 0x7
| ('''code''' >> 3) - intra_run['''run''']
|}
 
Codes for inter block types:
{| border="1" cellpadding="5" style="border-collapse: collapse; border-style: dashed; border-color: #2f6fab;"
|- bgcolor="#f0f0f0" |
! code !! run !! value
|-
| 0 || 0 || 0
|-
| 1 || 0 || 1
|-
| 2 || 1 || 1
|-
| 3 || 2 || 1
|-
| 4 || 0 || 2
|-
| 5 || 3 || 1
|-
| 6 || 4 || 1
|-
| 7 || 5 || 1
|-
| 8 || 0 || 3
|-
| 9 || 1 || 2
|-
| 10 || 2 || 2
|-
| 11 || 6 || 1
|-
| 12 || 7 || 1
|-
| 13 || 8 || 1
|-
| 14 || 9 || 1
|-
| 15 || 0 || 4
|-
| 16-... (inter)
| '''code''' & 0xF
| ('''code''' >> 4) - inter_run['''run''']
|}
 
Run correction values:
  intra_run = { 8, 2, 0, 0, 0, -1, -1, -1, [minus  ones] };
  inter_run = { 4, 2, 2, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, [zeroes] };
 
=== Macroblock transform and dequantization ===
 
Transform coefficients:
 
  13  17  1  7
  13  7  -1 -17
  13  -7  -1  17
  13 -17  1  -7
 
Dequantization is performed by multiplying every coefficient by the same value determined by quantizer.
In case if inter blocks first coefficient may be quantized slightly differently:
 
For some intra blocks ('''TODO''': condition):
    dc = 169 * 1538 * block[0]
For chroma blocks (?):
    dc = (svq3_dequant_coeff[Q] * (block[0] >> 3)) >> 1;
 
Quantizer table (from svq3.c)
 
  static const uint32_t svq3_dequant_coeff[32] = {
    3881,  4351,  4890,  5481,  6154,  6914,  7761,  8718,
    9781, 10987, 12339, 13828, 15523, 17435, 19561, 21873,
  24552, 27656, 30847, 34870, 38807, 43747, 49103, 54683,
  61694, 68745, 77615, 89113,100253,109366,126635,141533
  };
 
Dequantization formula (dc=0 if not defined otherwise):
  out = (coeff * svq3_dequant_coeff[Q] + dc + 0x80000) >> 20;
 
== Packetization ==
The first byte of a SVQ3 RTP packet indicates the packet type(s). This is after the standard RTP headers.
 
{| border="1" cellpadding="5" style="border-collapse: collapse; border-style: dashed; border-color: #2f6fab;"
|- bgcolor="#f0f0f0" |
! 1st byte value !! packet type
|-
| 0x40 || config
|-
| 0x20 || start
|-
| 0x10 || end
|}
 
Note that a packet may have more than one type. Config packets are in practice transmitted individually since they do not contain additional video data beyond extradata.
 
The second byte of a SVQ3 RTP packet is ignored. This is likely reserved for future use.
 
All subsequent bytes are payload data.
 
SVQ3 does not make use of SDP FMTP attributes to carry codec-specific extradata; rather, this is carried within the RTP itself in config packets. Also, SVQ3 decoders expect extradata to be prefixed with the marker bytes "SEQH", followed by another 4 bytes indicating the length of the extradata. Neither are provided by the payload, and must be inserted by the depacketizer prior to decoding. The rest of the payload data is standard SVQ3 extradata. (todo: explain 'standard svq3 extradata'?)
 
Start packets come with a new RTP timestamp, and config packets may be periodically re-transmitted before a keyframe.


End packets simply indicate that the payload data constitutes the remainder of a SVQ3 frame.


[[Category:Video Codecs]]
[[Category:Video Codecs]]
[[Category:Incomplete Video Codecs]]

Revision as of 11:24, 3 October 2010

Video codec apparently based on an early H.264 draft.

Decoding Process

This codec extensively uses Golomb coding.

Slice Header

TODO

Macroblock layer

Each macroblock starts with Golomb code signalling MB type:

  • 0 - skip block
  • 1-7 - inter 16x16 blocks, type specifies prediction direction
  • 8 and 33 - intra 4x4 block (type 33 means that there are no coefficients for this block)
  • 9-32 - intra 16x16 blocks, type is used to set prediction direction and coded block pattern

In any case coefficients are stored in 4x4 (sub)blocks.

Dezigzag pattern (from H.264):

 o-->o-->o   o
         |  /|
 o   o   o / o
 | / |   |/  |
 o   o   o   o
   /
 o-->o-->o-->o

Coefficient decoding

Each coefficient is stored as Golomb codeword, last bit is coefficient sign, code = 0 means end of nonzero coefficients.

Codes for block type 3:

code run value
0-2 0 code
3 1 1
4-... code & 0x3 ((code + 9) >> 2) - run

Codes for intra block types:

code run value
0 0 0
1 0 1
2 1 1
3 0 2
4 2 1
5 0 3
6 0 4
7 0 5
8 3 1
9 4 1
10 1 2
11 1 3
12 0 6
13 0 7
14 0 8
15 0 9
16-... code & 0x7 (code >> 3) - intra_run[run]

Codes for inter block types:

code run value
0 0 0
1 0 1
2 1 1
3 2 1
4 0 2
5 3 1
6 4 1
7 5 1
8 0 3
9 1 2
10 2 2
11 6 1
12 7 1
13 8 1
14 9 1
15 0 4
16-... (inter) code & 0xF (code >> 4) - inter_run[run]

Run correction values:

 intra_run = { 8, 2, 0, 0, 0, -1, -1, -1, [minus  ones] };
 inter_run = { 4, 2, 2, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, [zeroes] };

Macroblock transform and dequantization

Transform coefficients:

 13  17   1   7
 13   7  -1 -17
 13  -7  -1  17
 13 -17   1  -7

Dequantization is performed by multiplying every coefficient by the same value determined by quantizer. In case if inter blocks first coefficient may be quantized slightly differently:

For some intra blocks (TODO: condition):

   dc = 169 * 1538 * block[0]

For chroma blocks (?):

   dc = (svq3_dequant_coeff[Q] * (block[0] >> 3)) >> 1;

Quantizer table (from svq3.c)

 static const uint32_t svq3_dequant_coeff[32] = {
   3881,  4351,  4890,  5481,  6154,  6914,  7761,  8718,
   9781, 10987, 12339, 13828, 15523, 17435, 19561, 21873,
  24552, 27656, 30847, 34870, 38807, 43747, 49103, 54683,
  61694, 68745, 77615, 89113,100253,109366,126635,141533
 };

Dequantization formula (dc=0 if not defined otherwise):

 out = (coeff * svq3_dequant_coeff[Q] + dc + 0x80000) >> 20;

Packetization

The first byte of a SVQ3 RTP packet indicates the packet type(s). This is after the standard RTP headers.

1st byte value packet type
0x40 config
0x20 start
0x10 end

Note that a packet may have more than one type. Config packets are in practice transmitted individually since they do not contain additional video data beyond extradata.

The second byte of a SVQ3 RTP packet is ignored. This is likely reserved for future use.

All subsequent bytes are payload data.

SVQ3 does not make use of SDP FMTP attributes to carry codec-specific extradata; rather, this is carried within the RTP itself in config packets. Also, SVQ3 decoders expect extradata to be prefixed with the marker bytes "SEQH", followed by another 4 bytes indicating the length of the extradata. Neither are provided by the payload, and must be inserted by the depacketizer prior to decoding. The rest of the payload data is standard SVQ3 extradata. (todo: explain 'standard svq3 extradata'?)

Start packets come with a new RTP timestamp, and config packets may be periodically re-transmitted before a keyframe.

End packets simply indicate that the payload data constitutes the remainder of a SVQ3 frame.