FFmpeg Wishlist

From MultimediaWiki
Revision as of 04:57, 30 September 2007 by Vitor (talk | contribs) (Add and format links)
Jump to navigation Jump to search

Temporary FFmpeg wish/todo list. This list is deprecated, please do not add new items to this list, use instead the bugtracker wishlist or the TODO file in the source tree. Also, send a message to the mailing list before implementing one of those items. They could be work-in-progress or not wanted anymore.

FFmpeg module refactoring

Multimedia programs tend to be highly modular in design and FFmpeg is no exception. However, it does not make the best use of independent modules. The major task in refactoring FFmpeg modules will be to reorganize code so that each individual codec or muxer/demuxer module can be easily enabled and disabled at compile time. This task also entails creating a test suite that can automatically enable each module, one at a time, and validate that FFmpeg still builds and works.

Decoders

Demuxers

  • IFF demuxer (with anim and sound decoding)
    • mark cox (melbournemark at gmail dot com) is currently working on this.
    • xine has a demuxer/decoder for iff. Manfred Tremmel the author of the xine code has agreed to relicence his xine code as LGPL to allow easier use in ffmpeg. Thanks Manfred.
    • Werner Randelshofer the author of multishow has also granted permission for his iff code to be relicenced as LGPL. Thanks Werner.
    • Xanim also contains code to support iff, it may contain some formats not available in xine.
  • g723.1 / rtp demuxer
  • g729 / rtp demuxer
  • Vivo demuxer, look at the mplayer vivo demuxer for reference
  • XMV / FMV (Xbox Media Video) demuxer (from Microsoft and based on WMV8)
    • Open source and legal demuxer/decoder but copyrighted source/specification:

http://sourceforge.net/tracker/index.php?func=detail&aid=1097094&group_id=53761&atid=471491 also look at http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/25207/focus=25224 and http://www.maxconsole.net/?mode=news&newsid=411 for hints/tips

Muxers

  • DVB (MPEG-TS) muxer inside DVB containers
    • MPEG-1/2 video-streams inside DVB containers
    • MPEG-4 ASP video-streams inside DVB containers
    • MPEG-4 AVC (H.264) video-streams inside DVB containers
    • AC3 audio-streams inside DVB containers
      • Mutiple AC3 audio-streams inside DVB containers
    • MP3 audio-streams inside DVB containers
      • Mutiple MP3 audio-streams inside DVB containers
  • NSV muxer
  • NSA muxer

Microsoft DirectShow and DirectX and MediaFoundation

Native DirectShow support

Option to build FFmpeg decoder/encoder/demuxer/muxer and post-processing filters for the DirectShow API for Windows by Microsoft, (the native DirectX 8/9 Direct3D overlay for video playback), so that FFmpeg has native support to be compiled for DirectShow and thus be used directly by players that use DirectShow. It should be noted that there are already one very popular fork of FFmpeg available that has native DirectShow support and that is FFdshow, (a other DirectShow implementation of FFmpeg is available as "drffmpeg" which is part of the DrDivX project).

http://sourceforge.net/projects/drdivx/ (drffmpeg)

Native MediaFoundation support

Microsoft Windows Vista™ has new audio (EVR) and audio (SAR) renderers that FFmpeg needs output modules for

DirectX Video Acceleration (DXVA)

DirectX Video Acceleration (DXVA) 1.0 AND 2.0 support, (for GPU accelerated video decoding under Windows).
Note! Native DirectShow support in FFmpeg is before DirectX VA (DXVA) video decoding support can be added!

http://download.microsoft.com/download/5/b/9/5b97017b-e28a-4bae-ba48-174cf47d23cd/MED134_WH06.ppt

Features

  • Create a new audio API system
  • radix-4 fft routines
  • Grabbing from video devices under windows
    • Apply this VFW capture patch
    • Create a DirectShow patch
  • -[h|v]flip options for ffplay
  • Improved exition documentation and add additional means to document
    • Web
    • WIKI
    • manpage
  • Add XING and/or VBRI header parsing support to the MP3 decoder/parser (for VBR encoded audio files)
  • Add GAIN (MP3Gain) header parsing support to the MP3 decoder/parser
    • Also add GAIN (AACGain) header parsing support to the AAC decoder/parser

Subtitles

  • Create a common 'subtitles parser library' (and/or an API system for adding support for additional subtitle formats?) - a common sub-library to FFmpeg with all subtile decoders/demuxers/parsers gathered (similar to the libpostproc and libavutils). Call it "libsubs" (or "libsub", "libsubtitles" or whatever). Move FFmpeg's existing VobSub and DVBsub code there, so no matter if they are bitmap or text-based subs all existing and future subtile code is collected there. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional subtitles.
    • Maybe use MPlayer's recently added "libass" (SSA/ASS subtile reader) as a base for such a common library?
  • Support for advanced SSA/ASS rendering
    • Possible source are libass or the asa library
  • Support bold, italic, underline, RGB colors, size changes and font changes for a whole line or part of one line
  • Line 23 signal (a.k.a. "Wide-screen signal") detecting and use for DVD-Video (VobSub)
  • Support for the subtitles HTML tags
  • Capability of displaying subtitles with no video enabled (for example for audio-books)
  • Support for Karaoke subtitles (for kar and cdg, etc.)
  • Dual-subtitle-display (display two subtitles/languages at the same time, one at the bottom as normal plus one at the top of the screen)
  • Capability of moving the subtitles in the picture (freetype renderer)
  • Support more subtitle formats (text and bitmap-based):
    • Closed captioning (CC) subtile support - (Closed captions for the deaf and hard of hearing, also known as "Line 21 captioning", uses VobSub bitmaps)
      • xine have a SPU decoder for subpictures and Closed Captions software decoding
    • DirectVobSub (VSFilter) - standard VobSubs (DVD-Video subtitles) embedded in AVI containers
    • DivX Subtitles (XSUB) display/reader/decoder (Note: bitmap based subtitle, similar to VobSub)
    • SubRip (.srt) subtile support (Note: simple text-based based subtitle with timestamp)
    • Subviewer (.sub) subtile support (Note: simple text-based based subtitle with timestamp)
    • MicroDVD (.sub) subtile support (Note: simple text-based based subtitle with timestamp
    • Sami (.smi) subtile support (Note: simple text-based based subtitle with timestamp)
    • SubStation Alpha (.ssa+.ass) subtile support (Note: advanced text-based based subtitle with timestamps and XY location on screen)
    • RealText (.rt) subtile support
    • PowerDivx (.psb) subtile support
    • Universal Subtitle Format (.usf) subtile support
    • Structured Subtitle Format (.ssf) subtile support


Misc

  • Add a aac parser so -acodec copy to mp4/mov will work
  • Clean up the h263 rtp patch.

Streaming Media Network Protocols

Streaming Media Network Protocols (client and server-side) improvements/enhancements and related ideas for new features/functions.

  • Create a common 'stream demuxer/parser library' for the client-side to receive input streams (and/or API for adding support for additional streaming formats?) - a LGPL'ed sub-library in FFmpeg with all stream demuxers/parsers gathered (similar to the libpostproc and libavutil). Call it "libstream" (or "stream" or whatever). Move FFmpeg's existing stream code there like HTTP and RTSP/RTP. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional streaming formats. All togther making it super easy for audio/video players using FFmpeg to add all-in-one streaming support to their player.
    • Maybe use either MPlayer's "stream" library structure, LIVE555, cURL, or probebly the better libnms (from NeMeSi) as a base for such a common library?
  • Add support for additional streaming protocols (on the client side) and improve/enhance support for existing protocols:
    • HTTP (Hypertext Transfer Protocol) client
      • plus a SSL (Secure Sockets Layer) client support for HTTPS
    • UDP (User Datagram Protocol) client
    • RTSP - Real-Time Streaming Protocol (RFC2326) client
    • RTP/RTCP - Real-Time Transport Protocol/RTP Control Protocol (RFC3550) client
    • RTP Profile for Audio and Video Conferences with Minimal Control (RFC3551) client
    • RealMedia RTSP/RDT (Real Time Streaming Protocol / Real Data Transport) client
    • SDP (Service Discovery Protocol) / SSDP (Simple Service Discovery Protocol) client
    • MMS (Microsoft Media Services) client
      • including the subprotocol mmsh (MMS over HTTP) and mmst (MMS over TCP)
  • FFServer (streaming server) updating and improving:
    • FFServer code hasn't been update for quite a while
    • Support for RTSP interleaved RTP media
    • RTSP over HTTP tunneling
    • SLL (Secure Sockets Layer) support
    • TLS (Transport Layer Security) support
    • SCTP (Stream Control Transmission Protocol) support
      • including tunnel SCTP over UDP
    • Per-asset accounting options
    • Profiling and performance improvements of the RTSP, HTTP and RTP server code
    • Streaming to clients like WMP 9, 10 and 11 is broken
    • MMS server streaming support in FFServer, (especially for Linux).

See Also