FFmpeg codec HOWTO

From MultimediaWiki
Jump to navigation Jump to search

This page is meant as an introduction to the internal codec API in FFmpeg. It will also show how the codecs are connected with the demuxers. This is by no means a complete guide but enough to understand how to add a codec to FFmpeg.

libavcodec/avcodec.h

The first thing to look at is the AVCodec struct.

typedef struct AVCodec {
    const char *name;
    enum CodecType type;
    enum CodecID id;
    int priv_data_size;
    int (*init)(AVCodecContext *);
    int (*encode)(AVCodecContext *, uint8_t *buf, int buf_size, void *data);
    int (*close)(AVCodecContext *);
    int (*decode)(AVCodecContext *, void *outdata, int *outdata_size,
                  uint8_t *buf, int buf_size);
    int capabilities;
    struct AVCodec *next;
    void (*flush)(AVCodecContext *);
    const AVRational *supported_framerates; ///array of supported framerates, or NULL if any, array is terminated by {0,0}
    const enum PixelFormat *pix_fmts;       ///array of supported pixel formats, or NULL if unknown, array is terminanted by -1
} AVCodec;

Here we can see that we have some elements to name the codec, what type it is (audio/video), the supported pixel formats and some function pointers for init/encode/decode and close. Now lets see how it is used.

libavcodec/cook.c

If we look in this file at the bottom we can see this code:

AVCodec cook_decoder =
{
    .name = "cook",
    .type = CODEC_TYPE_AUDIO,
    .id = CODEC_ID_COOK,
    .priv_data_size = sizeof(COOKContext),
    .init = cook_decode_init,
    .close = cook_decode_close,
    .decode = cook_decode_frame,
};

First we get an AVCodec struct named cook_decoder. And then we set the variables of cook_decoder. Note that we only set the variables that are needed. Currently there is no encoder so we don't set any. If we now look at the id variable we can see that CODEC_ID_COOK isn't defined in libavcodec/cook.c. It is declared in avcodec.h.

libavcodec/avcodec.h

Here we will find the CodecID enumeration.

enum CodecID {
...
CODEC_ID_GSM,
CODEC_ID_QDM2,
CODEC_ID_COOK,
CODEC_ID_TRUESPEECH,
CODEC_ID_TTA,
...
};

CODEC_ID_COOK is there in the list. This is the list of all supported codecs in ffmpeg, the list is fixed and used internally to id every codec. Changeing the order would break binary compatibility.

This is all enough to declare a codec. Now we must register them for internal use also. This is done at runtime.


libavcodec/allcodecs.c

In this file we have the avcodec_register_all() function, it has entrys like this for all codecs.

...
#ifdef CONFIG_COOK_DECODER
   register_avcodec(&cook_decoder);
#endif //CONFIG_COOK_DECODER
...

The register_avcodec() call registers a codec for internal use. The defines around the register call are used so it is possible to not compile the decoder code for a specific codec. But what defines CONFIG_COOK_DECODER? This is extracted by configure with this command line:

grep 'register_avcodec(&[a-z]' libavcodec/allcodecs.c  | sed 's/.*&\(.*\)).*/\1/'

So adding a register_avcodec(&new_decoder) entry in allcodecs.c and reconfigure is enough to add the needed define. One more thing to note is that cook.c isn't included in allcodecs.c so the symbol cook_decoder can't be found. Thus it has to be declared somewhere, that happens in avcodec.h and it is declared as an external symbol.

Now we have everything to hockup a codec now we will see how the codec. For this we look into libavformat.

libavformat/rm.c

If we think of an imaginary rm file that ffmpeg is about to process, the first thing that happens is that it is identified as a rm file. It is passed on to the rm demuxer (rm.c). The rm demuxer looks through the file and finds out that it is a cook file.

...
} else if (!strcmp(buf, "cook")) {
st->codec->codec_id = CODEC_ID_COOK;
...

Now ffmpeg knows what codec to init and where to send the payload from the container. So back to cook.c and the initialization process.

libavcodec/cook.c Init

After ffmpeg knows what codec to use, it calls the declared initialization function pointer declared in the codecs AVCodec struct. In cook.c it is called cook_decode_init. Here we setup as much as we can before we start decoding. The following things should be handled in the init, vlc table initialization, table generation, memory allocation and extradata parsing.

libavcodec/cook.c Close

The cook_decode_close function is the codec clean-up call. All memory, vlc tables, etc. should be freed here.

libavcodec/cook.c Decode

In cook.c the name of the decode call is cook_decode_frame.

static int cook_decode_frame(AVCodecContext *avctx,
            void *data, int *data_size,
            uint8_t *buf, int buf_size) {
...

The function has 5 arguments:

  • avctx is a pointer to a AVCodecContext
  • data is the pointer to the outbuffer
  • data_size is a variable that should be set to the outbuffer size in bytes
  • buf is the pointer to the inbuffer
  • buf_size is the size of the inbuffer

The decode function shall return the amount of bytes consumed from the inbuffer.


That's how it works without to much detail.