Libavfilter

Reasons why a filter API is needed in FFmpeg

FFmpeg's already well-known libavcodec module has become the de facto standard library for video decoding and encoding in free software projects. Unfortunately, no similar standard library has surfaced for audio/video filtering and otherwise working with audio/video stream once it has been decoded. Various multimedia projects (such as MPlayer, Xine, GStreamer, VirtualDub, etc.) have implemented their own filter systems to various degrees of success. What is needed is a high quality audio and video filter API - efficient, flexible enough to meet all the requirements which have led various projects to invent their own filter system, and yet easy to use or develop new filters with. This proposal is to implement a high quality video API and filter library for FFmpeg, where it can be easily used by other multimedia-related software projects.

Possible known issues

• no frame rate estimation is supported (see post in -soc [1])
• make ffmpeg.c honor the aspect ratio outputted by the filter chain (thanks vdmsss for the bug report) -- Updated: probably fixed, but maybe not printing the right PAR in the terminal

Feature wish list

Warning: This is not an official wish list. Before implementing any one of these items, ask first in the ffmpeg-devel mailing-list.

• See the TODO file in SoC SVN tree
• Create a SDK (Software Development Kit) with templates for the A/V filter APIs
• Add support for audio filters
  • See FFmpeg audio API
• Add support for subtitle filters
  • Add a subtitle drawing support to simplify ffplay.c
• Create (or port) additional pre-process and post-process video filters to FFmpeg:
  • General post-proc sources are MPlayer (libmpcodecs vf_*.c filters), Xine, FFdshow, VLC, VirtualDub, GStreamer, foobar, and XMMS, vhook
  • More image scaling methods:
    • linear interpolation
    • bilinear interpolation
    • cubic interpolation
    • bicubic interpolation
    • trilinear interpolation
  • SSP (Statistical Post-Processing)
  • DeBlocking
  • DeRinging
  • IVTC
  • Sharpen / UnSharpen (Soften)
  • ReQuantization
  • Auto-Luminance
  • Blurring / DeNoising / Spatial Blur / Temporal Blur
  • Deinterlace (weave AND bob) filters
    • Possible sources: DScaler or y4mscale?
  • 2:3 pull-down / ivtc (inverse telecine) for 24 progressive-frames on 30 FPS TV's
    • Possible sources: DScaler or y4mscale?
  • NTSC => PAL, and PAL => NTSC frame-rate (FPS) adjust and reclock filter for NTSC <=> PAL conversion

See also

• FFmpeg filter howto
• FFmpeg audio API
• FFmpeg filtering Guide