Understanding VC-1

There is a WMV9/VC-1 reference decoder available on the internet which answers to the filename vc1_reference_decoder_release6.zip. This section of the MultimediaWiki is an effort to create an open VC-1 specification based on that reference implementation.

Acronyms

This section defines various terms that are useful to know when plodding through a discussion of the VC-1 reference implementation.

• AC/DC/DCAC = transform coefficients
• CBP/CBPCY = coded block pattern
• IC = intensity compensation
• IDU = independently decodable unit
• MB = macroblock
• MV = motion vector
• MVBP = motion vector block pattern
• NZC = non-zero coefficients
• SBP = sub block pattern
• TTBLK = ??? (may mean block transform type)
• TTMB = ??? (may mean macroblock transform type)
• VOP = ???

Hierarchy of VC-1 Decoding Functions

These functions are necessary for processing VC-1 data using the reference decoder:

// set up pointers to bitstream (extradata) in internal state structure
vc1DECBIT_InitialiseBitstream

// analyze how much memory the client app needs to allocate
vc1DEC_DecoderRequirements

// init decoder
vc1DEC_DecoderInitialise

// process the extradata setup bitstream
vc1DEC_DecodeSequence

// decode a frame
vc1DEC_DecodeFrame

These are the call trees for the above functions

vc1DECBIT_InitialiseBitstream
 +- vc1DECBIT_ReadBytes

vc1DEC_DecoderRequirements
 +- vc1DECBIT_GetBits
 +- vc1TOOLS_InitReferencePicture
   +- vc1TOOLS_InitImagePosition

vc1DEC_DecoderInitialise
 +- vc1DEC_SetMaxSize
 +- vc1TOOLS_InitReferencePicture

vc1DEC_DecodeSequence
 +- vc1DECSEQ_UnpackSequenceLayer
   +- vc1DECBIT_GetBits

vc1DEC_DecodeFrame
 +- vc1DECPIC_UnpackPictureLayer
   +- vc1DECPIC_ReadPictureLayer
     +- vc1DECBIT_GetBits
     +- vc1DECPIC_SetDimensionsInMB
     +- vc1DECPIC_UnpackPictureLayerAdvanced
       +- vc1DECBIT_GetBits
       +- vc1DECBIT_GetVLC
       +- vc1DECPIC_UnpackPanScanParams
         +- vc1DECBIT_GetBits        
     +- vc1DECPIC_UnpackPictureLayerSimpleMain
       +- vc1DECBIT_BitCountGet
       +- vc1DECBIT_GetBits
       +- vc1DECBIT_GetVLC
       +- vc1DECPIC_UnpackQuantizationParams
         +- vc1DECBIT_GetBits
         +- vc1IQUANT_GetQuantizer
       +- vc1DECPIC_SetDimensionsInMB
       +- vc1DECBITPL_ReadBitplane
         +- vc1DECBIT_GetBits
         +- vc1DECBIT_GetVLC
         +- vc1DECBITPL_DecodeNorm2Bits
           +- vc1DECBIT_GetBits
           +- vc1DECBIT_GetVLC
         +- vc1DECBITPL_DecodeNorm6Bits
           +- vc1DECBIT_GetBits
           +- vc1DECBIT_GetVLC
         +- vc1DECBITPL_DecodeRowskipBits
           +- vc1DECBIT_GetBits
         +- vc1DECBITPL_DecodeColskipBits
           +- vc1DECBIT_GetBits
         +- vc1DECBITPL_DecodeRawBits
       +- vc1DECPIC_UnpackVOPDQUANTParams
         +- vc1DECBIT_GetBits
   +- vc1DECPIC_UnpackFieldPictureLayer
     +- vc1DECPIC_ReadAdvancedPictureLayer
       +- vc1DECPIC_UnpackFieldPictureLayerIAdvanced
         +- vc1DECPIC_UnpackQuantizationParams
         +- vc1DECBIT_GetBits
         +- vc1DECBITPL_ReadBitplane
         +- vc1DECBIT_GetVLC
         +- vc1DECPIC_UnpackVOPDQUANTParams
       +- vc1DECPIC_UnpackFieldPictureLayerPBAdvanced
         +- vc1DECPIC_UnpackQuantizationParams
         +- vc1DECBIT_GetBits
         +- vc1DECBIT_GetVLC
         +- vc1DECPIC_UnpackInterlaceMVModeParams
         +- vc1DECBITPL_ReadBitplane
     +- vc1DECPIC_DisplayPicture
       +- vc1DECPIC_DisplayField
         +- vc1DECPIC_InitialiseAppPicture
         +- vc1TOOLS_ResolutionUpsample
     +- vc1TOOLS_InitReferencePicture
     +- vc1TOOLS_NewReference
     +- vc1TOOLS_CopyReference
     +- vc1TOOLS_ICPadReferencePicture
       +- vc1TOOLS_RangeReduceReference
         +- vc1TOOLS_RangeReduce16
         +- vc1TOOLS_RangeExpand
       +- vc1TOOLS_IntensityCompensate
         +- vc1TOOLS_ICComponent
       +- vc1TOOLS_PadReferencePicture
         +- vc1TOOLS_PadComponent
     +- vc1SCALEMV_InitScaleMV
     +- vc1DECSLICE_DecodeSlice
       +- vc1DECBIT_GetBits
       +- vc1DECPIC_UnpackSyncmarker
         +- vc1DECBIT_AlignBit
           +- vc1DECBIT_ReadBytes
         +- vc1DECBIT_GetBits
       +- vc1DECMB_UnpackMacroblockLayer
         +- vc1IQUANT_ChooseQuantizer
         +- vc1DECMB_UnpackMacroblockI
           +- vc1DECBITPL_ReadBitplaneBit
             +- vc1DECBIT_GetBits
           +- vc1DECBIT_GetVLC
           +- vc1PREDCBP_ApplyCBPCYPred
           +- vc1DECBIT_GetBits
           +- vc1DECMB_UnpackMBQuantParams
             +- vc1DECBIT_GetBits
         +- vc1DECMB_UnpackMacroblockProgP
           +- vc1DECBITPL_ReadBitplaneBit
           +- vc1DECMB_UnpackMacroblockProgPSkipped
             +- vc1DECBIT_LookBits
               +- vc1DECBIT_ReadBits
             +- vc1PREDMV_PredictProgressiveMV
               +- vc1PRED_pTopBlk
               +- vc1PRED_pLeftBlk
               +- vc1PRED_pB1MVBlk
               +- vc1PRED_pB4MVBlk
               +- vc1TOOLS_Median3
               +- vc1CROPMV_BPredPullBack
             +- vc1DECBIT_GetBits
             +- vc1DECMV_ApplyMVPrediction
             +- vc1DERIVEMV_DecideChromaBlockType
           +- vc1DECMB_UnpackMacroblockProgP1MV
             +- vc1DECMV_UnpackMVData
             +- vc1DECBIT_LookBits
             +- vc1PREDMV_PredictProgressiveMV
             +- vc1DECBIT_GetBits
             +- vc1DECMV_ApplyMVPrediction
             +- vc1DERIVEMV_DecideChromaBlockType
             +- vc1DECMB_UnpackMBQuantParams
             +- vc1DECBIT_GetVLC
             +- vc1DECMB_DecodeTransformInfo
               +- vc1DECMB_UnpackTTMB
                 +- vc1DECBIT_GetVLC
           +- vc1DECMB_UnpackMacroblockProgP4MV
             +- vc1DECBIT_GetVLC
             +- vc1DECMV_UnpackMVData
             +- vc1DECBIT_LookBits
             +- vc1PREDMV_PredictProgressiveMV
             +- vc1DECBIT_GetBits
             +- vc1DECMV_ApplyMVPrediction
             +- vc1DERIVEMV_DecideChromaBlockType
             +- vc1DECMB_UnpackMBQuantParams
             +- vc1PREDDCAC_ACPREDPresent
             +- vc1DECMB_DecodeTransformInfo
         +- vc1DECMB_UnpackMacroblockFieldP
           +- vc1DECBIT_GetVLC
           +- vc1DECMV_UnpackMVDataInterlace
             +- vc1DECBIT_GetVLC
             +- vc1DECBIT_GetBits
           +- vc1DECBIT_LookBits
           +- vc1PREDMV_PredictInterlacedFieldMV
             +- vc1SCALEMV_ScaleMV
             +- vc1TOOLS_Median3
           +- vc1DECBIT_GetBits
           +- vc1DECMV_ApplyMVPrediction
           +- vc1DERIVEMV_DecideChromaBlockType
           +- vc1DECMB_UnpackMBQuantParams
           +- vc1DECMB_DecodeTransformInfo
         +- vc1DECMB_UnpackMacroblockFrameP
           +- vc1DECBITPL_ReadBitplaneBit
           +- vc1PREDMV_PredictInterlacedFrameMV
             +- vc1PREDMV_FrameMBPred
             +- vc1TOOLS_Median3
           +- vc1DECMV_ApplyMVPrediction
           +- vc1DECBIT_GetVLC
           +- vc1DECBIT_GetBits
           +- vc1DERIVEMV_DecideChromaBlockType
           +- vc1DECMB_UnpackMBQuantParams
           +- vc1DECMV_UnpackMVDataInterlace
         +- vc1DECMB_UnpackMacroblockProgB
           +- vc1DECBITPL_ReadBitplaneBit
           +- vc1DECMV_UnpackMVData
           +- vc1DECBIT_GetVLC
           +- vc1DECMB_UnpackMBQuantParams
           +- vc1DECBIT_GetBits
           +- vc1PREDMV_PredictProgressiveMV
           +- vc1DECMV_ApplyMVPrediction
           +- vc1DERIVEMV_DecideChromaBlockType
           +- vc1DECMB_DecodeTransformInfo
         +- vc1DECMB_UnpackMacroblockFieldB
           +- vc1DECBIT_GetVLC
           +- vc1DECBITPL_ReadBitplaneBit
           +- vc1DECBITPL_SkipBitplaneBit
           +- vc1DECBIT_GetBits
           +- vc1DECMV_UnpackMVDataInterlace
           +- vc1PREDMV_PredictInterlacedFieldMV
           +- vc1DECMV_ApplyMVPrediction
           +- vc1DECMB_UnpackMBQuantParams
           +- vc1DERIVEMV_DecideChromaBlockType
           +- vc1DECMB_DecodeTransformInfo
         +- vc1DECMB_UnpackMacroblockFrameB
           +- vc1DECBITPL_ReadBitplaneBit
           +- vc1DECBIT_GetVLC
           +- vc1DECBITPL_SkipBitplaneBit
           +- vc1DECBIT_GetBits
           +- vc1DECMB_UnpackMBQuantParams
           +- vc1DERIVEMV_DecideChromaBlockType
           +- vc1DECMV_UnpackMVDataInterlace
           +- vc1PREDMV_PredictInterlacedFrameMV
           +- vc1DECMV_ApplyMVPrediction
         +- vc1DECMB_AssignCodedBlockPattern
         +- vc1DECBLK_DecodeBlockLayer
           +- vc1DECBLK_UnpackIntraBlock
             +- vc1PREDDCAC_DCDefault
               +- vc1PREDDCAC_DCStepSize
             +- vc1PREDDCAC_PredictDCAC
               +- vc1PREDDCAC_GetQuant
               +- vc1PREDDCAC_ScaleDC
                 +- vc1PREDDCAC_DCStepSize
             +- vc1DECBLK_UnpackDCDifferential
               +- vc1DECBIT_GetVLC
               +- vc1DECBIT_GetBits
             +- vc1DEC3DH_DecodeACRunLevel
               +- vc1DEC3DH_ChooseACCodingSet
               +- vc1DEC3DH_ChooseEscapeMode3Table
               +- vc1DEC3DH_UnpackTransformACCoef
                 +- vc1DECBIT_GetVLC
                 +- vc1DECBIT_GetBits
             +- vc1DECZZ_DeZigZagBlock
               +- vc1DECZZ_DeZigZag
             +- vc1DECBLK_ApplyACPrediction
             +- vc1PREDDCAC_CopyDCAC
           +- vc1DECBLK_UnpackInterBlock
             +- vc1DECBLK_UnpackSubBlockPattern
               +- vc1DECBIT_GetVLC
             +- vc1DECBLK_UnpackTTBLK
               +- vc1DECBIT_GetVLC
             +- vc1DEC3DH_DecodeACRunLevel
             +- vc1DECZZ_DeZigZagBlock
           +- vc1INTERP_PredictMB
             +- vc1DERIVEMV_StoreMotionVectors
             +- vc1DERIVEMV_DeriveMV
               +- vc1DERIVEMV_DirectMV
                 +- vc1DERIVEMV_DeriveProgMV
                   +- vc1TOOLS_Median3
                   +- vc1TOOLS_Median4
                 +- vc1CROPMV_ChromaPullBack
                 +- vc1DERIVEMV_DeriveIntFieldMV
                   +- vc1TOOLS_Median4
                   +- vc1TOOLS_Median3
                 +- vc1DERIVEMV_DeriveIntFrameDirectMV
                 +- vc1CROPMV_PPredPullBack
               +- vc1DERIVEMV_FillInInterlaceFieldMV
                 +- vc1PREDMV_PredictInterlacedFieldMV
                   +- vc1SCALEMV_ScaleMV
                   +- vc1TOOLS_Median3
             +- vc1INTERP_InterpolateBlock
               +- vc1CROPMV_LumaPullBack
               +- vc1DERIVEMV_DeriveChromaMV
                 +- vc1DERIVEMV_DeriveProgMV
                 +- vc1CROPMV_LumaPullBack
                 +- vc1DERIVEMV_DeriveSecondStageChromaMV
                   +- vc1CROPMV_ChromaPullBack
                 +- vc1DERIVEMV_DeriveIntFieldMV
                   +- vc1TOOLS_Median4
               +- vc1CROPMV_ChromaPullBack
               +- vc1INTERP_InterpPatchQuarterPelBilinear
               +- vc1INTERP_InterpPatchHalfPelBilinear
                 +- vc1INTERP_InterpPatchQuarterPelBilinear
               +- vc1INTERP_InterpPatchQuarterPelBicubic
                 +- vc1INTERP_CopyPatch
                 +- vc1INTERP_InterpPatchQuarterPelBicubicVert
                 +- vc1INTERP_InterpPatchQuarterPelBicubicHoriz
                 +- vc1INTERP_InterpPatchQuarterPelBicubicDiag
               +- vc1INTERP_InterpPatchHalfPelBicubic
                 +- vc1INTERP_InterpPatchQuarterPelBicubic
             +- vc1INTERP_AverageBlocks
           +- vc1RECON_ReconstructMB
             +- vc1IQUANT_InverseACQuantize
             +- vc1IQUANT_InverseDCQuantize
               +- vc1PREDDCAC_DCStepSize
             +- vc1ITRANS_InverseTransformBlock
               +- vc1ITRANS_InverseTransform_AnnexA1
             +- vc1RECON_ApplyPredictionAndCopyMB
               +- vc1INTERP_InterlaceDiffMB
               +- vc1INTERP_InterlacePredMB
               +- vc1SMOOTH_OverlapSmoothMB
                 +- vc1PRED_pLeftMB
                 +- vc1TOOLS_GetPictureDestination
                 +- vc1SMOOTH_OverlapSmoothVertBlk
                 +- vc1SMOOTH_OverlapSmoothHorizMB
                   +- vc1TOOLS_GetPictureDestination
                   +- vc1SMOOTH_OverlapSmoothHorizBlk
               +- vc1TOOLS_GetPictureDestination
               +- vc1SMOOTH_OverlapSmoothHorizMB
       +- vc1DEBLOCK_DeblockSlice
         +- vc1DEBLOCK_HorizDeblockMB
           +- vc1TOOLS_GetPictureDestination
           +- vc1DEBLOCK_HorizDeblockBlk
         +- vc1DEBLOCK_VertDeblockMB
           +- vc1TOOLS_GetPictureDestination
           +- vc1DEBLOCK_VertDeblockBlk
       +- vc1TOOLS_CopyReference
       +- vc1DECPIC_DisplayPicture

vc1DEC_DecoderRequirements

This function is an API glue function to the VC-1 reference decoder. The function takes the first part of the encoded bitstream, and DecoderCofiguration data structure, and returns the total number of bytes that the client application must allocate for the decoder to establish all of its internal data structures.

 2 bits: VC-1 profile
   0 = simple profile
   1 = main profile
   2 = reserved, should not occur
   3 = advanced profile
 if (profile == advanced)
   ...more processing goes here...
 else  // simple or main profile
   move data from original page here

vc1DEC_SetMaxSize

This function takes the width and height dimension pair for an image. The function validates that the number of macroblocks is allowed under to current profile. Then it assigns the dimension pair (not rounded) to 5 other dimension pairs:

• vc1DEC_sState.vc1_sSequenceLayer.MaxCoded[Width|Height]
• vc1DEC_sState.vc1_sSequenceLayer.Coded[Width|Height]
• vc1DEC_sState.vc1_sSequenceLayer.Display[Width|Height]
• vc1DEC_sState.vc1_sPosition.Coded[Width|Height]
• vc1DEC_sState.vc1_sPosition.MaxCoded[Width|Height]

vc1DECBIT_AlignBit

This function aligns the input bitstream on a byte boundary.

vc1DECBIT_BitCountGet

This function returns the total number of unconsumed bits remaining in the input bitstream.

vc1DECBIT_GetBits

GetBits returns and consumes the next n bits from the bitstream, as opposed to LookBits, which does not consume the bits after reading them.

vc1DECBIT_GetVLC

The GetVLC function returns the value represented by the next variable length code (i.e., Huffman) code in the bitstream. The reference decoder performes this operation in an exceptionally slow manner but it is also not held to any specific performance standards. Thus, a new VC-1 implementation should maintain VLC tables and decode values from then however it sees fit to do so.

vc1DECBIT_InitialiseBitstream

This function takes a pointer to bitstream accounting structure, a pointer to a buffer containing an encoded bitstream, and the length of that buffer. It initializes the internal bitstream accounting data structure. It also calls vc1DECBIT_ReadBytes to fill up the internal bit buffer in preparation for bit extraction. This function can also accept a flag that specifies the IDUs are encapsulated.

vc1DECBIT_LookBits

LookBits asks the bitstream reader to return the next n bits from the stream but to not actually consume them, as GetBits() would do.

vc1DECBIT_ReadBits

ReadBits is essentially equivalent to GetBits.

vc1DECBIT_ReadBytes

ReadBytes() is a bitstream accounting function that ensures the 32-bit bit buffer is completely filled.

vc1INTERP_AverageBlocks

This function takes pointers to 2 64-element byte arrays and averages them into a third 64-element array. The averaging weights up:

 average = (a + b + 1) / 2

vc1TOOLS_CopyReference

This function copies an entire reference picture and all associated data structures to another. The reference source notes that it is to be used in the case of a skipped frame.

vc1TOOLS_GetPictureDestination

Provided a block number, this function returns a data structure with the information about that block.

vc1TOOLS_Median3

The Median3 function takes 3 numbers as input and returns the middle value of the 3.

vc1TOOLS_Median4

The Median4 function takes 4 numbers as input and returns the average between the middle 2 values of the set, weighted down. For 4 inputs, (a, b, c, d), find the minimum and maximum values of the set (min, max) and the median4 value is computed as:

 (a + b + c + d - min - max) / 2

vc1TOOLS_RangeExpand

This function takes a scale value and a pointer to a rectangle of byte values and performs the following operation on each byte in the rectangle:

 byte = (((byte - 128) * scale + 4) / 8) + 128

The final arithmetic is saturated to an unsigned byte range (0..255).

vc1TOOLS_RangeReduce16

This function takes a pointer to a rectangle of byte values and performs the following operation on each byte:

 byte = ((byte - 128) / 2) + 128

vc1TOOLS_RangeReduceReference

This function performs intensity compensation on a reference picture. It takes a reference picture data structure and a new range reduction value. Then, for each of the planes, Y, U, and V:

 if range reduction value is 16 then
   call RangeReduce16 with the entire plane as the rectangle
 else
   call RangeExpand with the entire plane as the rectange
   and 16 as the scale value

note: if 16 is always the scale value, then there is an optimization opportunity in RangeExpand()

vc1TOOLS_ResolutionUpsample

The reference decoder includes the upsampling filter but the exact implementation is not defined in the official spec.

(TODO: Fill in the reference decoder's algorithm anyway)