FFmpeg Wishlist: Difference between revisions
Jump to navigation
Jump to search
Gamester17 (talk | contribs) |
Gamester17 (talk | contribs) |
||
Line 54: | Line 54: | ||
* improved documentation (web,manpage) | * improved documentation (web,manpage) | ||
== Subtitles == | |||
* Create a common 'subtitles parser library' (and/or API for adding support for additional subtitle formats?) - a common sub-library to FFmpeg with all subtile decoders/demuxers/parsers gathered (similar to the libpostproc and libavutils). Call it "libsubs" (or "libsub", "libsubtitles" or whatever). Move FFmpeg's existing VobSub and DVBsub code there, so no matter if they are bitmap or text-based subs all existing and future subtile code is collected there. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional subtitles. | * Create a common 'subtitles parser library' (and/or an API system for adding support for additional subtitle formats?) - a common sub-library to FFmpeg with all subtile decoders/demuxers/parsers gathered (similar to the libpostproc and libavutils). Call it "libsubs" (or "libsub", "libsubtitles" or whatever). Move FFmpeg's existing VobSub and DVBsub code there, so no matter if they are bitmap or text-based subs all existing and future subtile code is collected there. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional subtitles. | ||
**Maybe use MPlayer's recently added "'''libass'''" (SSA/ASS subtile reader) as a base for such a common library? | **Maybe use MPlayer's recently added "'''libass'''" (SSA/ASS subtile reader) as a base for such a common library? | ||
* Support for advanced SSA/ASS rendering | * Support for advanced SSA/ASS rendering | ||
* Support bold, italic, underline, RGB colors, size changes and font changes for a whole line or part of one line | **Possible source are libass or the asa library | ||
* Support bold, italic, underline, RGB colors, size changes and font changes for a whole line or part of one line | |||
* [http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=284541 Line 23 signal (a.k.a. "Wide-screen signal")] detecting and use for DVD-Video (VobSub) | * [http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=284541 Line 23 signal (a.k.a. "Wide-screen signal")] detecting and use for DVD-Video (VobSub) | ||
* Support for the subtitles HTML tags | * Support for the subtitles HTML tags | ||
* Capability of displaying subtitles with no video enabled (for example for audio-books) | * Capability of displaying subtitles with no video enabled (for example for audio-books) | ||
* Support for Karaoke subtitles (for kar and cdg, etc.) | * Support for Karaoke subtitles (for kar and cdg, etc.) | ||
* Dual-subtitle-display (display two subtitles/languages at the same time, one at the bottom as normal plus one at the top of the screen) | |||
* Capability of moving the subtitles in the picture (freetype renderer) | * Capability of moving the subtitles in the picture (freetype renderer) | ||
*Support more subtitle formats (text and bitmap-based): | *Support more subtitle formats (text and bitmap-based): | ||
** [http://en.wikipedia.org/wiki/Closed_captioning Closed captioning (CC)] subtile support - (Closed captions for the deaf and hard of hearing, also known as "Line 21 captioning", uses VobSub bitmaps) | ** [http://en.wikipedia.org/wiki/Closed_captioning Closed captioning (CC)] subtile support - (Closed captions for the deaf and hard of hearing, also known as "Line 21 captioning", uses VobSub bitmaps) | ||
***[http://www.xinehq.de xine] have a SPU decoder for subpictures and Closed Captions software decoding | |||
** DirectVobSub (VSFilter) - standard VobSubs (DVD-Video subtitles) embedded in AVI containers | ** DirectVobSub (VSFilter) - standard VobSubs (DVD-Video subtitles) embedded in AVI containers | ||
** [http://en.wikipedia.org/wiki/DivX#DivX_Subtitles_.28XSUB.29 DivX Subtitles (XSUB)] display/reader/decoder (Note: bitmap based subtitle, similar to VobSub) | ** [http://en.wikipedia.org/wiki/DivX#DivX_Subtitles_.28XSUB.29 DivX Subtitles (XSUB)] display/reader/decoder (Note: bitmap based subtitle, similar to VobSub) |
Revision as of 13:53, 20 March 2007
A temporary FFmpeg wish/todo list:
Decoders
- g723.1/rtp decoder
- g729/rtp decoder
- Bethsoft VID decoder
- a Monkey's Audio decoder, look at the c++ sdk sources
- a JPEG2000 decoder
- a gsm decoder
- QCELP decoder spec is c.s0020 and source is c.r0020
- AMV decoder, http://scrub50187.com/ has the creator. wikipedia has articles about the format also.
- integer only vorbis decoder (to replace tremor)
- LGPL'ed LE-AAC and HE-AAC decoder
- aacPlus (a.k.a. AAC+) decoder (Note: aacPlus v1 is HE-AAC + SBR, aacPlus v2 is HE-AAC + SBR + PS. Standard HE-AAC decoders can decode aacPlus encoded files/streams but without SBR and PS you do not get the full quality. By adding a SBR decoder and a PS decoder to FFmpeg and coupling it with an existing HE-AAC decoder you could playback aacPlus at full quality? A SBR decoder could be shared with a mp3PRO decoder as mp3PRO also uses SBR).
- SBR (Spectral Band Replication) decoder
- PS (Parametric Stereo) decoder
- mp3PRO decoder (Note: mp3PRO is MP3 + SBR. Standard MP3 decoders can decode mp3PRO encoded files/streams but without SBR you do not get the full quality. By adding a SBR decoder to FFmpeg and coupling it with the existing MP3 decoder you could playback mp3PRO at full quality? A SBR decoder could be shared with a aacPlus decoder as aacPlus also uses SBR).
- SBR (Spectral Band Replication) decoder
- PS (Parametric Stereo) decoder
- MPEG Surround decoder/parser (for all audio but especially MP3/mp3PRO and AAC/aacPlus as those are in use today). MPEG Surround technology share similar characteristics with SBR (Spectral Band Replication) and PS (Parametric Stereo), which mp3PRO and aacPlus also use, so if SBR and PS decoders was added to FFmpeg then those could probebly share common code with a MPEG Surround decoder/parser.
- SBR (Spectral Band Replication) decoder
- PS (Parametric Stereo) decoder
- Add j-type picture support to wmv8 decoder
- MLP decoder
- add b-frame support to ffsvq3
- Indeo 4 and 5 decoder
- vnc decoder, files created by vncrec. reuse code from VMware Video decoder. http://www.sodan.org/~penny/vncrec/
Demuxers
- iff demuxer (with anim and sound decoding), xine has a decoder for it
- g723.1/rtp demuxer
- g729/rtp demuxer
- Bethsoft VID demuxer
- vivo demuxer, look at the mplayer vivo demuxer for reference
- a xmv demuxer, look at http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/25207/focus=25224 and http://www.maxconsole.net/?mode=news&newsid=411 for hints; also look at http://sourceforge.net/tracker/index.php?func=detail&aid=1097094&group_id=53761&atid=471491 (should be a working demuxer/decoder)
- AMV demuxer, http://scrub50187.com/ has the creator. wikipedia has articles about the format also.
libstream (a common 'stream' library)
- Create a common 'stream demuxer/parser library' (and/or API for adding support for additional streaming formats?) - a LGPL'ed sub-library in FFmpeg with all stream demuxers/parsers gathered (similar to the libpostproc and libavutil). Call it "libstream" (or "stream" or whatever). Move FFmpeg's existing stream code there like HTTP and RTSP/RTP. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional streaming formats. All togther making it super easy for audio/video players using FFmpeg to add all-in-one streaming support to their player.
- Maybe use either MPlayer's "stream" library, LIVE555, or probebly the better libnms (from NeMeSi) as a base for such a common library?
- HTTP (Hypertext Transfer Protocol)
- UDP (User Datagram Protocol)
- RTSP - Real-Time Streaming Protocol (RFC2326)
- RTP/RTCP - Real-Time Transport Protocol/RTP Control Protocol (RFC3550)
- RTP Profile for Audio and Video Conferences with Minimal Control (RFC3551)
- RealMedia RTSP/RDT (Real Time Streaming Protocol / Real Data Transport) proprietary transport protocol developed by RealNetworks to stream RealVideo/RealAudio
- SDP (Service Discovery Protocol) / SSDP (Simple Service Discovery Protocol)
- MMS (Microsoft Media Services)
Features
- a new audio api
- radix-4 fft routines
- grabbing from video devices under windows, apply this vfw capture patch http://lists.mplayerhq.hu/pipermail/ffmpeg-user/2006-December/005607.html
- -[h|v]flip options for ffplay
- improved documentation (web,manpage)
Subtitles
- Create a common 'subtitles parser library' (and/or an API system for adding support for additional subtitle formats?) - a common sub-library to FFmpeg with all subtile decoders/demuxers/parsers gathered (similar to the libpostproc and libavutils). Call it "libsubs" (or "libsub", "libsubtitles" or whatever). Move FFmpeg's existing VobSub and DVBsub code there, so no matter if they are bitmap or text-based subs all existing and future subtile code is collected there. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional subtitles.
- Maybe use MPlayer's recently added "libass" (SSA/ASS subtile reader) as a base for such a common library?
- Support for advanced SSA/ASS rendering
- Possible source are libass or the asa library
- Support bold, italic, underline, RGB colors, size changes and font changes for a whole line or part of one line
- Line 23 signal (a.k.a. "Wide-screen signal") detecting and use for DVD-Video (VobSub)
- Support for the subtitles HTML tags
- Capability of displaying subtitles with no video enabled (for example for audio-books)
- Support for Karaoke subtitles (for kar and cdg, etc.)
- Dual-subtitle-display (display two subtitles/languages at the same time, one at the bottom as normal plus one at the top of the screen)
- Capability of moving the subtitles in the picture (freetype renderer)
- Support more subtitle formats (text and bitmap-based):
- Closed captioning (CC) subtile support - (Closed captions for the deaf and hard of hearing, also known as "Line 21 captioning", uses VobSub bitmaps)
- xine have a SPU decoder for subpictures and Closed Captions software decoding
- DirectVobSub (VSFilter) - standard VobSubs (DVD-Video subtitles) embedded in AVI containers
- DivX Subtitles (XSUB) display/reader/decoder (Note: bitmap based subtitle, similar to VobSub)
- SubRip (.srt) subtile support (Note: simple text-based based subtitle with timestamp)
- Subviewer (.sub) subtile support (Note: simple text-based based subtitle with timestamp)
- MicroDVD (.sub) subtile support (Note: simple text-based based subtitle with timestamp
- Sami (.smi) subtile support (Note: simple text-based based subtitle with timestamp)
- SubStation Alpha (.ssa+.ass) subtile support (Note: advanced text-based based subtitle with timestamps and XY location on screen)
- RealText (.rt) subtile support
- PowerDivx (.psb) subtile support
- Universal Subtitle Format (.usf) subtile support
- Structured Subtitle Format (.ssf) subtile support
- Closed captioning (CC) subtile support - (Closed captions for the deaf and hard of hearing, also known as "Line 21 captioning", uses VobSub bitmaps)
Misc
- NSV muxer
- add PAFF to H.264 decoder
- clean up alac decoder
- integrate speex, glue code or native
- add xing or vbri header parsing support to the mp3 decoder/parser
- add GAIN (MP3Gain) header parsing support to the MP3 decoder/parser
- also add GAIN (AACGain) header parsing support to the AAC decoder/parser
- a aac parser so -acodec copy to mp4/mov works
- clean up the h263 rtp patch found on this page: http://www.salyens.com/downloads/index.html#ffmpeg-0.4.7
- add nice error messages to the flv demuxer(Nelly Moser)
- amr decoder
- dv encoder/decoder
- game formats:
Snow
- multiple reference frames improvements
- decide which frames to keep (e.g. long-term refs)
- some changes to the mv prediction code
- non translational motion compensation
- estimate non translational parameters per block by using surrounding motion vectors
- add a ac coded bit per block to switch between translational and non-translational MC
- borrow the non translational MC code from libmpcodecs/vf_perspective.c
- some changes to the encoder to decide between translational and non t.
- Trellis quantization (select quantized coefficient so as to minimize the rate distrortion
- 4x4 sized block support (we have 16x16 and 8x8 currently)
- 1/8 pel motion compensation / estimation support (pretty much just encoder changes needed which in case of the iterative me should be trivial)
- improve the intra color decision
A/V Filters
- Adopt MPlayer's filter system or create new one.
- See http://article.gmane.org/gmane.comp.video.ffmpeg.devel/39130 for michaelni's idea of what to do
- Decide on name of a such filter API.
- libavfilter (conflicts with LAVF)? libavmunge?
- Create (or port) additional pre-process and post-process video filters:
- General post-proc sources are MPlayer (libmpcodecs vf_*.c filters) and FFdshow
- SSP (Statistical-Post-Processing)
- DeBlocking
- DeRinging
- Sharpen / UnSharpen (Soften)
- ReQuantization
- Auto-Luminance
- Blurring / DeNoising
- Deinterlace (weave AND bob) filters
- 2:3 pull-down / ivtc (inverse telecine) for 24 progressive-frames on 30 FPS TV's
- NTSC => PAL, and PAL => NTSC frame-rate (FPS) adjust and reclock filter for NTSC <=> PAL conversion
- NTSC <=> PAL frame-rate adjust FPS ratios?: 23.97 <=> 25, 24 <=> 25, 30 <=> 25, 25 <=> 30
- Create (or port) additional pre-process and post-process audio filters:
- Audio re-sampler (sample rate converter) filter
- Possible source is SRC (Secret Rabbit Code)
- Audio re-sampler (sample rate converter) filter