IBM UltiMotion: Difference between revisions

From MultimediaWiki
Jump to navigation Jump to search
m (point to mirror directory, for consistency)
 
(6 intermediate revisions by one other user not shown)
Line 1: Line 1:
* FourCC: ULTI
* FourCC: ULTI
* Company: [[IBM]]
* Company: [[IBM]]
* Official Description: [http://multimedia.cx/UMSPEC.ZIP Ultimotion Digital Video Data Stream Specification]
* Official Description: [http://multimedia.cx/mirror/UMSPEC.ZIP Ultimotion Digital Video Data Stream Specification]
* Additional Documentation: [http://multimedia.cx/ultimotion-format.txt 'Description of the IBM UltiMotion (ULTI) Video Codec' by Mike Melanson and Konstantin Shishkov ].''
* Additional Documentation: [http://multimedia.cx/ultimotion-format.txt 'Description of the IBM UltiMotion (ULTI) Video Codec' by Mike Melanson and Konstantin Shishkov ].''
* Samples: [http://samples.mplayerhq.hu/V-codecs/ULTI/ http://samples.mplayerhq.hu/V-codecs/ULTI/]
* Samples: [http://samples.mplayerhq.hu/V-codecs/ULTI/ http://samples.mplayerhq.hu/V-codecs/ULTI/]
Line 10: Line 10:
== Decoding Algorithm ==
== Decoding Algorithm ==


The Ultimotion algorithm operates natively on a scaled-down [[Raw YUV|YUV]] 4:1:0 planar colorspace. This means that for each 4x4 block of pixels (quadrant), each of the 16 pixels has a luminance (Y) sample and the entire block shares one chrominance sample (expressed as two color difference components : C<sub>b</sub> (or U) and C<sub>r</sub> (or V). The colorspace is actually non-linear quantized to fixed values from 2 tables. Indices into these tables (64 possible values for luminance and 16 for each chrominance component) are transmitted in the final bitstream.
The Ultimotion algorithm operates natively on a scaled-down [[Raw YUV|YUV]] 4:1:0 planar colorspace. This means that for each 4x4 block of pixels (quadrant), each of the 16 pixels has a luminance (Y) sample and the entire block shares one chrominance sample expressed as two color difference components : C<sub>b</sub> (or U) and C<sub>r</sub> (or V). The colorspace is actually non-linear quantized to fixed values from 2 tables. Indices into these tables (64 possible values for luminance and 16 for each chrominance component) are transmitted in the final bitstream.


The Ultimotion algorithm codes a frame by dividing it into a series of 8x8 blocks. Each 8x8 block is subdivided and coded as 4 4x4 quadrants. The 4 quadrants, 0-3, are coded in the following arrangement:
The Ultimotion algorithm codes a frame by dividing it into a series of 8x8 blocks. Each 8x8 block is subdivided and coded as 4 4x4 quadrants. The 4 quadrants, 0-3, are coded in the following arrangement:
Line 60: Line 60:
       coding mode 1:
       coding mode 1:
         paint the luminance samples using a gradient pattern based on one  
         paint the luminance samples using a gradient pattern based on one  
           6-bit sample coded in the stream
           sample coded in the stream
   
   
         get next byte from stream (b1)
         get next byte from stream (b1)
Line 74: Line 74:
   
   
       coding mode 2:
       coding mode 2:
         paint the luminance samples using a gradient pattern based on  
         paint the luminance samples using a gradient pattern based on either
           either 4 6-bit samples coded in the stream or 4 6-bit samples
           four samples retrieved from a 4,096-entry codebook using an index
          retrieved from a 16,384-entry codebook using an index coded in  
          coded in the stream or four samples coded in the stream  
          the stream
         if mode is 0
         if mode is 0
           get the next 2 bytes from the stream as a big endian 16-bit  
           get the next 2 bytes from the stream as a big endian 16-bit  
Line 83: Line 82:
           gradient fill pattern = bits 15-12
           gradient fill pattern = bits 15-12
           luminance codebook index = bits 11-0
           luminance codebook index = bits 11-0
           Y0..Y3 are set according to the codebook entry (see References
           Y0..Y3 are set according to the codebook entry (see section Luminance
             for Ultimotion luminance codebook)
             Codebook Enumeration)
           if bit 3 of the gradient fill pattern is set, swap Y0 with Y3  
           if bit 3 of the gradient fill pattern is set
            and Y1 with Y2; clear bit 3 of the gradient fill pattern
            swap Y0 with Y3 and Y1 with Y2
            clear bit 3 of the gradient fill pattern
           fill luminance samples according to Y0..Y3 and the gradient fill  
           fill luminance samples according to Y0..Y3 and the gradient fill  
             pattern (see section Gradient Fill Patterns)
             pattern (see section Gradient Fill Patterns)
         else (mode is non-zero)
         else (mode is non-zero)
           get the next 3 bytes from the stream as a big endian 24-bit  
           get the next 3 bytes from the stream as a big endian 24-bit number:
            number:
            Y0 = bits 23-18
          Y0 = bits 23-18
            Y1 = bits 17-12
          Y1 = bits 17-12
            Y2 = bits 11-6
          Y2 = bits 11-6
            Y3 = bits 5-0
          Y3 = bits 5-0
           fill luminance samples according to Y0..Y3 and gradient fill  
           fill luminance samples according to Y0..Y3 and gradient fill  
             pattern 8 (see section Gradient Fill Patterns)
             pattern C (see section Gradient Fill Patterns)
   
   
       coding mode 3:
       coding mode 3:
         flexible coding mode that can paint the luminance samples with 16
         flexible coding mode that can paint the luminance samples with an
           6-bit samples coded in the stream, or with a gradient fill  
           arbitrary pattern using two samples and a set of flags coded in the  
          pattern and 4 6-bit luminance samples coded in the stream, or  
          stream, or with a gradient fill pattern and four samples coded in the  
          with an arbitrary pattern using 2 luminance samples and a set of
          stream, or with 16 samples coded in the stream
          flags coded in the stream
         if mode is 0
         if mode is 0
          get the next byte in the stream (b1)
           if bit 7 of b1 is 0
           if bit 7 of b1 is set
            gradient fill pattern = bits 6-4 of b1
            get the next byte in the stream (b1)
            Y0 = ((b1 & 0x0F) << 2) | (b1 >> 6)
            Y1 = bits 5-0 of b1
            Y2 = next byte in stream & 0x3F
            Y3 = next byte in stream & 0x3F
            fill luminance samples according to Y0..Y3 and the gradient
              fill pattern (see section Gradient Fill Patterns)
          else
             flagsA = b1
             flagsA = b1
             flagsB = next byte in the stream
             flagsB = next byte in the stream
Line 131: Line 119:
               for example, if bit 3 of flagsB is 1, Y1 would be used in  
               for example, if bit 3 of flagsB is 1, Y1 would be used in  
               the lower left corner of the quadrant, else Y0
               the lower left corner of the quadrant, else Y0
          else
            gradient fill pattern = bits 6-4 of b1
            get the next byte in the stream (b2)
            Y0 = ((b1 & 0x0F) << 2) | (b2 >> 6)
            Y1 = bits 5-0 of b2
            Y2 = next byte in stream & 0x3F
            Y3 = next byte in stream & 0x3F
            fill luminance samples according to Y0..Y3 and the gradient
              fill pattern (see section Gradient Fill Patterns)
         else (mode is non-zero)
         else (mode is non-zero)
           get the next 16 6-bit numbers from the stream as the 16  
           get the next 12 bytes from the stream as 16 6-bit numbers as the 16  
             luminance samples for the quadrant, thus consuming 96 bits,
             luminance samples (Y0..Y15) for the quadrant
             or 12 bytes from the stream
          fill luminance samples according to Y0..Y15 and gradient fill
             pattern D (see section Gradient Fill Patterns)
   
   
       finally, output quadrant:
       finally, output quadrant:
Line 147: Line 145:
== Gradient Fill Patterns ==
== Gradient Fill Patterns ==


The ULTI codec has 11 gradient fill patterns for painting 4x4 sub blocks of luminance data using 4 luminance samples.
The ULTI codec has 12 gradient fill patterns for painting 4x4 sub blocks of luminance data using 4 (or 16) luminance samples.


* Gradient Pattern 0: (0 degrees)
* Gradient Pattern 0: (0 degrees)
Line 214: Line 212:
   Y2 Y2 Y3 Y3
   Y2 Y2 Y3 Y3
   Y2 Y2 Y3 Y3
   Y2 Y2 Y3 Y3
* Gradient Pattern D: (16 samples)
  Y0  Y1  Y2  Y3
  Y4  Y5  Y6  Y7
  Y8  Y9  Y10 Y11
  Y12 Y13 Y14 Y15
== Luminance Codebook Enumeration ==
The following pseudo-code creates the Ultimotion Luminance Transition Coding codebook
i = 0
for Y0 = 0 to 61
  for Y3 = Y0 + 2 to 63
    Yd = Y3 - Y0
    if Yd in (2,3,5,6,7,8,11,14,17,20)
      LTC[i++] = (Y0, Y0 + Yd/3, Y3 - Yd/3, Y3)
    endif
    if Yd in (4,5,6,7,8,11,14,17,20,23,26,29,32,36)
      LTC[i++] = (Y0, Y0 + Yd/2, Y3 - Yd/4, Y3)
      LTC[i++] = (Y0, Y0 + Yd/4, Y3 - Yd/4, Y3)
      LTC[i++] = (Y0, Y0 + Yd/4, Y3 - Yd/2, Y3)
    endif
    if Yd in (6,8,11,14,17,20,23,26,29,32,35,40,46)
      LTC[i++] = (Y0, Y3, Y3, Y3)
      LTC[i++] = (Y0, Y0, Y3, Y3)
      LTC[i++] = (Y0, Y0, Y0, Y3)
    endif
  next Y3
next Y0
It should be noted there is an error in the "Ultimotion Digital Video Data Stream Specification" - the pre-determined linear transitions are for Y deltas of 2, 3, 5, 6, 7, 8, 11, 14, 17, 20 (not 4). (If 4 is included then there are 4156 transitions. The various transitions are handled by the low contrast edge transitions.)


== Converting Ultimotion Scaled YUV To Correct YUV ==
== Converting Ultimotion Scaled YUV To Correct YUV ==

Latest revision as of 08:21, 25 October 2007

IBM Ultimotion is a video codec typically seen in AVI files, though there is no reason it could not be used in other container formats. Full codecs were available for Windows 3.1 (VfW), OS/2 and AIX. It is primarily a vector quantizing algorithm with simple inter-frame coding (unchanged block). Ultimotion encoding is significantly more computationally intensive than decoding.

Decoding Algorithm

The Ultimotion algorithm operates natively on a scaled-down YUV 4:1:0 planar colorspace. This means that for each 4x4 block of pixels (quadrant), each of the 16 pixels has a luminance (Y) sample and the entire block shares one chrominance sample expressed as two color difference components : Cb (or U) and Cr (or V). The colorspace is actually non-linear quantized to fixed values from 2 tables. Indices into these tables (64 possible values for luminance and 16 for each chrominance component) are transmitted in the final bitstream.

The Ultimotion algorithm codes a frame by dividing it into a series of 8x8 blocks. Each 8x8 block is subdivided and coded as 4 4x4 quadrants. The 4 quadrants, 0-3, are coded in the following arrangement:

 0 3
 1 2

Each block can be encoded using a single chrominance sample or unique chrominance sample for each quadrant. Each quadrant has 16 luminance samples that can be coded using a variety of methods.

To decode a frame of Ultimotion data, follow this process:

initialize uniq and mode flags to 0

for each 8x8 block in the frame, iterating from left -> right, top -> bottom:

 get the next byte in the stream (b0)
 if (b0 & 0xF8) == 0x70, handle escape code:
   if b0 is 0x70
     mode = next byte in stream
     note: only mode values of 0 or 1 are valid
   if b0 is 0x71
     uniq flag = -1 
   if b0 is 0x72
     toggle uniq flag (0/1)
   if b0 is 0x73
     end of frame, stop decoding
   if b0 is 0x74
     get the next byte in stream (b1)
     skip next (b1) blocks in the output frame (unchanged from previous frame)

 else
   if b0 is non-zero and uniq flag is 0
     chrominance byte = next byte from stream

   for each 4x4 quadrant in the block, arranged in the quadrant pattern:

     extract quadrant coding mode from b0:
       bits 7-6: quadrant 0 coding mode
       bits 5-4: quadrant 1 coding mode
       bits 3-2: quadrant 2 coding mode
       bits 1-0: quadrant 3 coding mode

     if coding mode is non-zero and uniq flag is non-zero
       chrominance byte = next byte from stream

     coding mode 0:
       skip quadrant (unchanged from the previous frame)

     coding mode 1:
       paint the luminance samples using a gradient pattern based on one 
         sample coded in the stream

       get next byte from stream (b1)
       bits 7-6 of b1 indicate gradient fill pattern:
         0 = gradient fill pattern A
         1 = gradient fill pattern 2
         2 = gradient fill pattern 6
         3 = gradient fill pattern B
       bits 5-0 of b1 indicate luminance sample Y0
         Y1 = Y0, Y2 = Y0 + 1, Y3 = Y0 + 1
       fill luminance samples according to Y0..Y3 and the gradient fill 
         pattern (see section Gradient Fill Patterns)

     coding mode 2:
       paint the luminance samples using a gradient pattern based on either 
         four samples retrieved from a 4,096-entry codebook using an index
         coded in the stream or four samples coded in the stream 
       if mode is 0
         get the next 2 bytes from the stream as a big endian 16-bit 
           number:
         gradient fill pattern = bits 15-12
         luminance codebook index = bits 11-0
         Y0..Y3 are set according to the codebook entry (see section Luminance
           Codebook Enumeration)
         if bit 3 of the gradient fill pattern is set
           swap Y0 with Y3 and Y1 with Y2
           clear bit 3 of the gradient fill pattern
         fill luminance samples according to Y0..Y3 and the gradient fill 
           pattern (see section Gradient Fill Patterns)
       else (mode is non-zero)
         get the next 3 bytes from the stream as a big endian 24-bit number:
           Y0 = bits 23-18
           Y1 = bits 17-12
           Y2 = bits 11-6
           Y3 = bits 5-0
         fill luminance samples according to Y0..Y3 and gradient fill 
           pattern C (see section Gradient Fill Patterns)

     coding mode 3:
       flexible coding mode that can paint the luminance samples with an 
         arbitrary pattern using two samples and a set of flags coded in the 
         stream, or with a gradient fill pattern and four samples coded in the 
         stream, or with 16 samples coded in the stream
       if mode is 0
         if bit 7 of b1 is 0
           flagsA = b1
           flagsB = next byte in the stream
           Y0 = next byte in stream & 0x3F
           Y1 = next byte in stream & 0x3F
           the 16 bits between flagsA and flagsB represent the pattern 
             used to place either Y0 or Y1 into the luminance grid

             A7 A6 A5 A4
             A3 A2 A1 A0
             B7 B6 B5 B4
             B3 B2 B1 B0

             for example, if bit 3 of flagsB is 1, Y1 would be used in 
             the lower left corner of the quadrant, else Y0
         else
           gradient fill pattern = bits 6-4 of b1
           get the next byte in the stream (b2)
           Y0 = ((b1 & 0x0F) << 2) | (b2 >> 6)
           Y1 = bits 5-0 of b2
           Y2 = next byte in stream & 0x3F
           Y3 = next byte in stream & 0x3F
           fill luminance samples according to Y0..Y3 and the gradient 
             fill pattern (see section Gradient Fill Patterns)
       else (mode is non-zero)
         get the next 12 bytes from the stream as 16 6-bit numbers as the 16 
           luminance samples (Y0..Y15) for the quadrant
         fill luminance samples according to Y0..Y15 and gradient fill 
           pattern D (see section Gradient Fill Patterns)

     finally, output quadrant:
       bits 7-4 of the chrominance byte represent the Cr (or V) component
         while bits 3-0 represent the Cb (U) component
       convert the 16 luminance values and the 2 chrominance values to a 
         proper YUV colorspace using the conversion tables in Appendix A

 if uniq flag is -1
   uniq flag = 0

Gradient Fill Patterns

The ULTI codec has 12 gradient fill patterns for painting 4x4 sub blocks of luminance data using 4 (or 16) luminance samples.

  • Gradient Pattern 0: (0 degrees)
 Y0 Y1 Y2 Y3
 Y0 Y1 Y2 Y3
 Y0 Y1 Y2 Y3
 Y0 Y1 Y2 Y3
  • Gradient Pattern 1: (22.5 degrees)
 Y1 Y2 Y3 Y3
 Y0 Y1 Y2 Y3
 Y0 Y1 Y2 Y3
 Y0 Y0 Y1 Y2
  • Gradient Pattern 2: (45 degrees)
 Y1 Y2 Y3 Y3
 Y1 Y2 Y2 Y3
 Y0 Y1 Y1 Y2
 Y0 Y0 Y1 Y2
  • Gradient Pattern 3: (67.5 degrees)
 Y2 Y3 Y3 Y3
 Y1 Y2 Y2 Y3
 Y0 Y1 Y1 Y2
 Y0 Y0 Y0 Y1
  • Gradient Pattern 4: (90 degrees)
 Y3 Y3 Y3 Y3
 Y2 Y2 Y2 Y2
 Y1 Y1 Y1 Y1
 Y0 Y0 Y0 Y0
  • Gradient Pattern 5: (112.5 degrees)
 Y3 Y3 Y3 Y2
 Y3 Y2 Y2 Y1
 Y2 Y1 Y1 Y0
 Y1 Y0 Y0 Y0
  • Gradient Pattern 6: (135 degrees)
 Y3 Y3 Y2 Y2
 Y3 Y2 Y1 Y1
 Y2 Y2 Y1 Y0
 Y1 Y1 Y0 Y0
  • Gradient Pattern 7: (157.5 degrees)
 Y3 Y3 Y2 Y1
 Y3 Y2 Y1 Y0
 Y3 Y2 Y1 Y0
 Y2 Y1 Y0 Y0
  • Gradient Pattern A: (homogeneous)
 Y0 Y0 Y0 Y0
 Y0 Y0 Y0 Y0
 Y0 Y0 Y0 Y0
 Y0 Y0 Y0 Y0
  • Gradient Pattern B: (270 degrees)
 Y0 Y0 Y0 Y0
 Y1 Y1 Y1 Y1
 Y2 Y2 Y2 Y2
 Y3 Y3 Y3 Y3
  • Gradient Pattern C: (subsampled)
 Y0 Y0 Y1 Y1
 Y0 Y0 Y1 Y1
 Y2 Y2 Y3 Y3
 Y2 Y2 Y3 Y3
  • Gradient Pattern D: (16 samples)
 Y0  Y1  Y2  Y3
 Y4  Y5  Y6  Y7
 Y8  Y9  Y10 Y11
 Y12 Y13 Y14 Y15

Luminance Codebook Enumeration

The following pseudo-code creates the Ultimotion Luminance Transition Coding codebook

i = 0
for Y0 = 0 to 61
  for Y3 = Y0 + 2 to 63
    Yd = Y3 - Y0
    if Yd in (2,3,5,6,7,8,11,14,17,20)
      LTC[i++] = (Y0, Y0 + Yd/3, Y3 - Yd/3, Y3)
    endif
    if Yd in (4,5,6,7,8,11,14,17,20,23,26,29,32,36)
      LTC[i++] = (Y0, Y0 + Yd/2, Y3 - Yd/4, Y3)
      LTC[i++] = (Y0, Y0 + Yd/4, Y3 - Yd/4, Y3)
      LTC[i++] = (Y0, Y0 + Yd/4, Y3 - Yd/2, Y3)
    endif
    if Yd in (6,8,11,14,17,20,23,26,29,32,35,40,46)
      LTC[i++] = (Y0, Y3, Y3, Y3)
      LTC[i++] = (Y0, Y0, Y3, Y3)
      LTC[i++] = (Y0, Y0, Y0, Y3)
    endif
  next Y3
next Y0

It should be noted there is an error in the "Ultimotion Digital Video Data Stream Specification" - the pre-determined linear transitions are for Y deltas of 2, 3, 5, 6, 7, 8, 11, 14, 17, 20 (not 4). (If 4 is included then there are 4156 transitions. The various transitions are handled by the low contrast edge transitions.)

Converting Ultimotion Scaled YUV To Correct YUV

To convert Ultimotion 6-bit luminance values to proper luminance samples, use the following conversion table:

unsigned char ulti_lumas[64] =
   { 0x10, 0x13, 0x17, 0x1A, 0x1E, 0x21, 0x25, 0x28,
     0x2C, 0x2F, 0x33, 0x36, 0x3A, 0x3D, 0x41, 0x44,
     0x48, 0x4B, 0x4F, 0x52, 0x56, 0x59, 0x5C, 0x60,
     0x63, 0x67, 0x6A, 0x6E, 0x71, 0x75, 0x78, 0x7C,
     0x7F, 0x83, 0x86, 0x8A, 0x8D, 0x91, 0x94, 0x98,
     0x9B, 0x9F, 0xA2, 0xA5, 0xA9, 0xAC, 0xB0, 0xB3,
     0xB7, 0xBA, 0xBE, 0xC1, 0xC5, 0xC8, 0xCC, 0xCF,
     0xD3, 0xD6, 0xDA, 0xDD, 0xE1, 0xE4, 0xE8, 0xEB};
     

To convert Ultimotion 4-bit chrominance values to proper chrominance samples, use the following conversion table:

unsigned char ulti_chromas[16] =
   { 0x60, 0x67, 0x6D, 0x73, 0x7A, 0x80, 0x86, 0x8D,
     0x93, 0x99, 0xA0, 0xA6, 0xAC, 0xB3, 0xB9, 0xC0};

References