https://wiki.multimedia.cx/api.php?action=feedcontributions&user=Gatoatigrado&feedformat=atomMultimediaWiki - User contributions [en]2024-03-29T07:17:15ZUser contributionsMediaWiki 1.39.5https://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_Of_Code_2007&diff=7686FFmpeg Summer Of Code 20072007-04-12T03:52:11Z<p>Gatoatigrado: /* Bethsoft VID */</p>
<hr />
<div>Google is sponsoring their third annual [http://code.google.com/soc/ Summer of Code] for the summer of 2007. This entails sponsoring students to work on assorted open source projects as well as sponsoring mentors in those same projects. Everyone wins.<br />
<br />
[[FFmpeg]] was a Summer of Code participant in the [http://code.google.com/soc/2006/ffmpeg/about.html summer of 2006] (here is the [[FFmpeg Summer Of Code|corresponding Wiki page]]).<br />
<br />
[[User:Multimedia Mike|Mike Melanson]] (mike -at- multimedia.cx) is the administrator and main point of contact for matters relating to the FFmpeg Summer of Code.<br />
<br />
== How to apply ==<br />
<br />
Before you can apply make sure you are qualified enough to apply. Last year 50% of the applicants weren't qualified for the task they applied for.<br />
<br />
* You have to know how to program in C fairly well.<br />
* We would like you to submit a patch that fixes a bug or adds a feature to FFmpeg. By doing that we will know that you are qualified for the task or not. On this page there is a list of [[Summer Of Code 2007#Qualification_tasks|Qualification Tasks]] that can be done. But you are free to submit anything you feel might be of value to FFmpeg. The qualification task can be done after you have filed you application (up until around April 7 as the list of accepted students is scheduled to be posted by Google on April 9).<br />
* Submit a good application through the formal Google Summer of Code process during the application timeframe (March 14-24, 2007).<br />
* You have to have >35 hours per week to put into the project.<br />
* You can't have another job at the same time as the SoC project.<br />
<br />
== Current Status ==<br />
* March 5-12, 2007: Application period for mentoring organizations.<br />
* March 6, 2007: Mike Melanson submitted FFmpeg mentoring application.<br />
* March 14, 2007: Google are now accepting student Applications.<br />
* March 15, 2007: FFmpeg got accepted as mentoring organization; now accepting student applications until March 24, 2007. Note that this is '''NOT''' the deadline to complete a qualification task; final student selections are to be made by April 9.<br />
* April 12, 2007: Google has allocated 7 project slots to FFmpeg.<br />
<br />
== Project Proposals ==<br />
<br />
=== Overview ===<br />
Qualifications for a good Summer of Code proposal:<br />
* discrete, well-defined, modular<br />
* comprised of a series of measurable sub-goals<br />
* based on open specs that are available free of charge<br />
* based on complete specs<br />
An example of a good proposal is the implementation of a decoder or demuxer for an as yet unsupported multimedia format, or an encoder or muxer for a format that can already be demuxed/decoded by FFmpeg.<br />
<br />
An example of a less desirable proposal is one that's not as measurable, such as refactoring APIs. Bad proposals tend to be ones that would require touching a lot of core code.<br />
<br />
To re-iterate:<br />
* Localized/isolated code projects = ''good''<br />
* Global code refactoring = ''bad''<br />
<br />
=== Note ===<br />
THIS LIST IS NOT THE PROPOSALS WE ARE SUBMITTING!<br />
<br />
Most of this list is just some ideas we are kicking around.<br />
<br />
=== Projects with Mentors (official projects) ===<br />
<br />
==== QCELP Decoder ====<br />
<br />
* Specification: QCELP decoder [http://www.3gpp2.org/Public_html/specs/alltsgscfm.cfm spec] is c.s0020 and source is c.r0020<br />
* Sample files: http://samples.mplayerhq.hu/A-codecs/qclp/<br />
<br />
''Mentor: Benjamin Larsson''<br />
<br />
==== Matroska Muxer ====<br />
<br />
* Specification: http://www.matroska.org/technical/specs/index.html<br />
* Sample files: http://samples.mplayerhq.hu/Matroska/<br />
<br />
''Mentor: Aurelien Jacobs; Backup mentors: Steve Lhomme, Ronald S. Bultje''<br />
<br />
==== MPEG TS/DVB Muxer ====<br />
* Specification: ISO 13818-1<br />
<br />
''Mentor: Baptiste Coudurier''<br />
<br />
==== MXF Muxer ====<br />
* Specification: SMPTE 377M<br />
<br />
''Mentor: Baptiste Coudurier''<br />
<br />
==== RV40 Decoder ====<br />
* [[RealVideo 4]] is steadily being reverse engineered and should be a reasonable candidate for re-implementation by the summer.<br />
<br />
''Mentor: Mike Melanson''<br />
<br />
==== PAFF decoding for H.264/AVC ====<br />
* Specification: [http://www.itu.int/rec/T-REC-H.264-200503-I/en ITU-T]<br />
* Sample files: http://samples.mplayerhq.hu/V-codecs/h264/PAFF/<br />
<br />
''Mentor: Loren Merritt''<br />
<br />
==== Dirac Encoder AND Decoder ====<br />
* Website: http://dirac.sf.net<br />
* Specification: http://dirac.sourceforge.net/specification.html<br />
<br />
''Mentor: Luca Barbato''<br />
<br />
==== E-AC3 Decoder ====<br />
* Specification: http://www.atsc.org/standards/a_52b.pdf<br />
* Samples: http://samples.mplayerhq.hu/evob/MAININTRO.EVO<br />
<br />
''Mentor: Justin Ruggles''<br />
<br />
<br />
<br />
=== Ideas for more projects to be determined ===<br />
<br />
==== JPEG2000 ====<br />
* Specifications: [http://www.itu.int/rec/T-REC-T/e As ITU-T recommendations], [http://isotc.iso.org/livelink/livelink/fetch/2000/2489/Ittf_Home/PubliclyAvailableStandards.htm ISO publicly available standards]<br />
* Sources: [http://www.ece.uvic.ca/~mdadams/jasper/ JasPer], [http://www.openjpeg.org/ OpenJpeg], [http://jj2000.epfl.ch/ JJ2000 (in Java)]<br />
* Samples: http://samples.mplayerhq.hu/jpeg2000/<br />
<br />
==== Monkey's Audio Decoder (APE) ====<br />
* Sources: [http://www.monkeysaudio.com/files/MAC_SDK_399.zip original sources], [http://sourceforge.net/projects/mac-port/ original sources port for non-win32 platforms], [http://jmac.sourceforge.net/ LGPLed Java implementation]<br />
<br />
==== Finish LC-AAC decoder and implement HE-AAC decoder (LGPL) ====<br />
* LGPL code: http://svn.mplayerhq.hu/aac/<br />
* GPL decoder (libfaad2): http://www.audiocoding.com<br />
* Possible AAC+ sub-project:<br />
** [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] decoder (aacPlus v1 support)<br />
** [http://www.codingtechnologies.com/products/paraSter.htm PS (Parametric Stereo)] decoder (for aacPlus v2 support)<br />
<br />
==== GSM Decoder ====<br />
* Specification + sample implementation: http://kbs.cs.tu-berlin.de/~jutta/toast.html<br />
* Samples: http://samples.mplayerhq.hu/A-codecs/GSM/<br />
<br />
==== i263 Decoder ====<br />
* Specification: [[I263 | I263 Format Specification at MultimediaWiki]]<br />
* Sources: [http://multimedia.cx/I263Src.zip GPLed I263 decoder]<br />
* Sample files: http://samples.mplayerhq.hu/V-codecs/I263/<br />
<br />
==== VP6 Encoder ====<br />
* Specification: [[On2 VP6]]<br />
* Samples: http://samples.mplayerhq.hu/V-codecs/VP6/<br />
<br />
==== NUT Muxer ====<br />
* General improvements and enhancements<br />
<br />
==== DPX/Cineon Encoder AND Decoder ====<br />
* Specification: SMPTE 268M<br />
* http://en.wikipedia.org/wiki/DPX<br />
* Sources: [http://cinepaint.cvs.sourceforge.net/cinepaint/cinepaint-project/cinepaint/plug-ins/cineon/ CinePaint implementation] , [http://linuxmovies.org/ linuxmovies.org list of open source film tools]<br />
* Samples: ftp://ftp.graphicsmagick.org/pub/dpx/<br />
<br />
==== OpenEXR Encoder AND Decoder ====<br />
* Website and open source tools: http://openexr.com/<br />
* License: modified BSD<br />
* http://en.wikipedia.org/wiki/OpenEXR<br />
* Sources: [http://cinepaint.cvs.sourceforge.net/cinepaint/cinepaint-project/OpenEXR/ CinePaint implementation] , [http://linuxmovies.org/ linuxmovies.org list of open source film tools]<br />
<br />
==== HD Photo Encoder AND Decoder ====<br />
* Sources and specification: http://www.microsoft.com/whdc/xps/hdphotodpk.mspx<br />
* http://en.wikipedia.org/wiki/HD_Photo<br />
<br />
==== mp3PRO & aacPlus & MPEG Surround decoders ====<br />
* [http://www.codingtechnologies.com/products/mp3pro.htm mp3PRO] decoder. Note: mp3PRO decoding means MP3 + [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] demuxing/decoding. (Standard MP3 decoders can decode mp3PRO encoded files/streams but without [http://www.codingtechnologies.com/products/sbr.htm SBR] you do not get the full quality. By adding a [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder to FFmpeg and coupling it with the existing MP3 decoder you could playback mp3PRO at full quality? A [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder could be shared with a aacPlus decoder as aacPlus also uses [http://www.codingtechnologies.com/products/sbr.htm SBR]).<br />
* [http://www.codingtechnologies.com/products/aacPlus.htm aacPlus] (a.k.a. AAC+) decoder. Note: aacPlus v1 decoding means HE-AAC + [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] demuxing/decoding, and aacPlus v2 decoding means HE-AAC + [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] + [http://www.codingtechnologies.com/products/paraSter.htm PS (Parametric Stereo)] demuxing/decoding. (Standard HE-AAC decoders can decode aacPlus encoded files/streams but without [http://www.codingtechnologies.com/products/sbr.htm SBR] and [http://www.codingtechnologies.com/products/paraSter.htm PS] you do not get the full quality. By adding a [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder and a [http://www.codingtechnologies.com/products/paraSter.htm PS] decoder to FFmpeg and coupling it with an existing HE-AAC decoder you could playback aacPlus at full quality? A [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder could be shared with a mp3PRO decoder as mp3PRO also uses [http://www.codingtechnologies.com/products/sbr.htm SBR]).<br />
* [http://www.mpegsurround.com MPEG Surround] decoder/parser (for all audio but especially MP3/mp3PRO and AAC/aacPlus as those are in use today). [http://www.codingtechnologies.com/products/mpgsrnd.htm MPEG Surround technology] share similar characteristics with [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] and [http://www.codingtechnologies.com/products/paraSter.htm PS (Parametric Stereo)] demuxing/decoding, which [http://www.codingtechnologies.com/products/mp3pro.htm mp3PRO] and [http://www.codingtechnologies.com/products/aacPlus.htm aacPlus] decoders also use, so if SBR and PS decoders was added to FFmpeg then those could probebly share common code with a [http://www.mpegsurround.com MPEG Surround] decoder/parser. ([http://www.divx.com DivX Inc.] is one company that uses MPEG Surround technology to achieve 5.1 channel surround sound in smaller files).<br />
<br />
==== Native DirectShow support ====<br />
Option to build FFmpeg decoder/encoder/demuxer/muxer and post-processing filters for the DirectShow API for Windows by Microsoft, (the native DirectX 8/9 Direct3D overlay for video playback), so that FFmpeg has native support to be compiled for DirectShow and thus be used directly by players that use DirectShow.<br />
* http://en.wikipedia.org/wiki/DirectShow<br />
http://en.wikipedia.org/wiki/Ffdshow<br />
* Specifications: http://msdn2.microsoft.com/en-us/library/ms783323.aspx<br />
**http://msdn2.microsoft.com/en-us/library/ms788119.aspx<br />
* Sources: http://sourceforge.net/projects/ffdshow-tryout/<br />
<br />
==== DirectX Video Acceleration (DXVA) 1.0 AND 2.0 for video decoding ====<br />
Support Microsoft DirectX VA (DXVA) API nativly for GPU assisted decoding under Windows. <br />
Note! For this, native support for the above mentioned DirectShow API is needed first.<br />
* http://en.wikipedia.org/wiki/DXVA<br />
* DXVA 1.0 (DirectX SDK) specifications: http://msdn2.microsoft.com/en-us/library/ms798379.aspx<br />
* DXVA 2.0 (Windows SDK) specifications: http://msdn2.microsoft.com/en-us/library/ms788119.aspx<br />
http://download.microsoft.com/download/5/b/9/5b97017b-e28a-4bae-ba48-174cf47d23cd/MED134_WH06.ppt<br />
<br />
==== Additional subtitle support ====<br />
* Create a common 'subtitles parser library' (and/or an API system for adding support for additional subtitle formats?) - a common sub-library to FFmpeg with all subtile decoders/demuxers/parsers gathered (similar to the libpostproc and libavutils). Call it "libsubs" (or "libsub", "libsubtitles" or whatever). Move FFmpeg's existing VobSub and DVBsub code there, so no matter if they are bitmap or text-based subs all existing and future subtile code is collected there. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional subtitles. <br />
**Maybe use MPlayer's recently added "'''libass'''" (SSA/ASS subtile reader) as a base for such a common library?<br />
* Support for advanced SSA/ASS rendering<br />
**Possible source are libass or the asa library <br />
* Support bold, italic, underline, RGB colors, size changes and font changes for a whole line or part of one line<br />
* [http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=284541 Line 23 signal (a.k.a. "Wide-screen signal")] detecting and use for DVD-Video (VobSub)<br />
* Support for the subtitles HTML tags<br />
* Capability of displaying subtitles with no video enabled (for example for audio-books)<br />
* Support for Karaoke subtitles (for kar and cdg, etc.)<br />
* Dual-subtitle-display (display two subtitles/languages at the same time, one at the bottom as normal plus one at the top of the screen)<br />
* Capability of moving the subtitles in the picture (freetype renderer)<br />
*Support more subtitle formats (text and bitmap-based):<br />
** [http://en.wikipedia.org/wiki/Closed_captioning Closed captioning (CC)] subtile support - (Closed captions for the deaf and hard of hearing, also known as "Line 21 captioning", uses VobSub bitmaps)<br />
***[http://www.xinehq.de xine] have a SPU decoder for subpictures and Closed Captions software decoding<br />
** DirectVobSub (VSFilter) - standard VobSubs (DVD-Video subtitles) embedded in AVI containers<br />
** [http://en.wikipedia.org/wiki/DivX#DivX_Subtitles_.28XSUB.29 DivX Subtitles (XSUB)] display/reader/decoder (Note: bitmap based subtitle, similar to VobSub)<br />
** [http://en.wikipedia.org/wiki/SubRip SubRip (.srt)] subtile support (Note: simple text-based based subtitle with timestamp)<br />
** Subviewer (.sub) subtile support (Note: simple text-based based subtitle with timestamp)<br />
** MicroDVD (.sub) subtile support (Note: simple text-based based subtitle with timestamp<br />
** [http://en.wikipedia.org/wiki/Sami_%28subtitle_tool%29 Sami (.smi)] subtile support (Note: simple text-based based subtitle with timestamp)<br />
** [http://en.wikipedia.org/wiki/SubStation_Alpha SubStation Alpha (.ssa+.ass)] subtile support (Note: advanced text-based based subtitle with timestamps and XY location on screen)<br />
** [http://en.wikipedia.org/wiki/Synchronized_Multimedia_Integration_Language RealText (.rt)] subtile support<br />
** PowerDivx (.psb) subtile support<br />
** [http://en.wikipedia.org/wiki/Universal_Subtitle_Format Universal Subtitle Format (.usf)] subtile support<br />
** Structured Subtitle Format (.ssf) subtile support<br />
<br />
==== libstream (a common 'stream client' library) ====<br />
* Create a common 'stream demuxer/parser library' for the client-side (and/or API for adding support for additional streaming formats?) - a LGPL'ed sub-library in FFmpeg with all stream demuxers/parsers gathered (similar to the libpostproc and libavutil). Call it "libstream" (or "stream" or whatever). Move FFmpeg's existing stream code there like HTTP and RTSP/RTP. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional streaming formats. All togther making it super easy for audio/video players using FFmpeg to add all-in-one streaming support to their player.<br />
**Maybe use either [http://www.mplayerhq.hu MPlayer]'s "''stream''" library structure, [http://www.live555.com LIVE555], or probebly the better [http://streaming.polito.it/client/library libnms] (from [http://streaming.polito.it/client NeMeSi]) as a base for such a common library?<br />
*Add support for additional streaming protocols (on the client side) and improve/enhance support for existing protocols:<br />
** HTTP (Hypertext Transfer Protocol) client<br />
** UDP (User Datagram Protocol) client<br />
** RTSP - Real-Time Streaming Protocol (RFC2326) client<br />
** RTP/RTCP - Real-Time Transport Protocol/RTP Control Protocol (RFC3550) client<br />
** RTP Profile for Audio and Video Conferences with Minimal Control (RFC3551) client<br />
** RealMedia RTSP/RDT (Real Time Streaming Protocol / Real Data Transport) client<br />
** SDP (Service Discovery Protocol) / SSDP (Simple Service Discovery Protocol) client<br />
** MMS (Microsoft Media Services) client<br />
<br />
== Qualification tasks ==<br />
<br />
Add a note if you choose to work on a Qualification task to avoid duplicate work.<br />
<br />
=== Quicktime IMA ADPCM encoder ===<br />
* Specification: ffmpeg decoder source, libavcodec/adpcm.c<br />
* Samples: http://samples.mplayerhq.hu/A-codecs/ima-adpcm/<br />
<br />
=== TIFF encoder ===<br />
* Specification: http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf <br />
* Samples: http://samples.mplayerhq.hu/mov/tiff/<br />
''Kamil Nowosad is working on this''<br />
<br />
''Bartlomiej Wolowiec is working on this''<br />
<br />
=== Vivo demuxer ===<br />
* Specification: look at the [[MPlayer]] [[vivo]] demuxer [http://svn.mplayerhq.hu/mplayer/trunk/libmpdemux/demux_viv.c?revision=22301&view=markup] <br />
* Samples: http://samples.mplayerhq.hu/vivo/<br />
''Alex Kalouguine is working on this task.''<br />
<br />
=== IFF/8SVX 8-bit audio demuxer ===<br />
* Specification: http://netghost.narod.ru/gff/vendspec/iff/iff.txt, http://sox.sourceforge.net/AudioFormats-11.html, and [http://xine.cvs.sourceforge.net/xine/xine-lib/src/demuxers/demux_iff.c?view=markup xine demuxer] <br />
* Samples: http://aminet.net/mods/smpl/<br />
<br />
''Tyler Williams is working on this task.''<br />
<br />
=== Port SGI image support to new API ===<br />
* Code: http://svn.mplayerhq.hu/ffmpeg/trunk/libavformat/sgi.c?revision=7867&view=markup&pathrev=7972<br />
FFmpeg changed image format APIs, but the SGI file format was never ported to the new API.<br />
**patch pending on ffmpeg-devel<br />
<br />
=== Optimize some code ===<br />
Do you think some code in FFmpeg could be made to run faster? We always love to get faster decoders or encoders. Note that this will require some ASM (assembly) skills and using timer code to benchmark and compare. Please make sure that any new code do not break compiling for other platforms.<br />
<br />
Speedups via optimizations (like SIMD for [http://en.wikipedia.org/wiki/3DNow%21 3DNow], [http://en.wikipedia.org/wiki/MMX MMX/MMX2], [http://en.wikipedia.org/wiki/Streaming_SIMD_Extensions SSE/SSE2/SSE3] and [http://en.wikipedia.org/wiki/AltiVec AltiVec]) are needed in FFmpeg's:<br />
* [[H.264]] video decoder<br />
* [[VC-1]] video decoder<br />
* [[AAC|AAC (LE-AAC and HE-AAC)]] audio decoder<br />
* cat libavcodec/*.c | grep -i optimize for more files that need optimization.<br />
**''Andrew Savchenko is working on this''<br />
<br />
=== BFI Playback System ===<br />
Add FFmpeg playback capability for the [[BFI]] format. This entails writing a new file demuxer and simple video decoder.<br />
<br />
''DrV said he was working on this.''<br />
<br />
=== THP Playback System ===<br />
Add FFmpeg playback capability for the [[THP]] format. This entails writing a new file demuxer and leveraging existing JPEG and ADPCM decoders to handle the video and audio data inside.<br />
<br />
''Marco Gerards is working on this (one patch applied, one patch pending)''<br />
<br />
=== Bethsoft VID ===<br />
Add FFmpeg playback capability for the [[Bethsoft VID]] format (new demuxer, new video decoder).<br />
: patch applied. --Nicholas<br />
<br />
=== Other Game Formats ===<br />
Several game formats are documented in this Wiki, but not yet implemented in FFmpeg. Investigate via the [[:Category:Game Formats]] page.<br />
<br />
=== Theora in Matroska ===<br />
The current mkv demuxer supports Vorbis but not Theora. Add support for Theora. This requires parsing the matroska extradata to extract the three header packets, and correctly passing these to the Theora decoder. You might want to read [http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2006-September/046115.html this thread] on the MPlayer list:<br />
<br />
''David Conrad is working on this''<br />
<br />
==See Also==<br />
*[[FFmpeg Wishlist]] for more tasks or ideas.<br />
*[[FFmpeg Summer Of Code|FFmpeg's Google SoC (Summer of Code) 2006 list of tasks]]<br />
*[http://bugzilla.mplayerhq.hu/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=FFmpeg&content= FFmpeg bugs] for ffmpeg bugs.</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_codec_HOWTO&diff=7587FFmpeg codec HOWTO2007-04-07T19:20:24Z<p>Gatoatigrado: /* FFmpeg demuxer connection */</p>
<hr />
<div>This page is meant as an introduction to the internal codec API in [[FFmpeg]].<br />
It will also show how the codecs are connected with the demuxers. This is by<br />
no means a complete guide but enough to understand how to add a codec to FFmpeg.<br />
[[RealAudio cook|Cook]] is used as an example throughout.<br />
<br />
== registering the codec ==<br />
<br />
=== libavcodec/avcodec.h ===<br />
The first thing to look at is the AVCodec struct.<br />
<br />
typedef struct AVCodec {<br />
const char *name;<br />
enum CodecType type;<br />
enum CodecID id;<br />
int priv_data_size;<br />
int (*init)(AVCodecContext *);<br />
int (*encode)(AVCodecContext *, uint8_t *buf, int buf_size, void *data);<br />
int (*close)(AVCodecContext *);<br />
int (*decode)(AVCodecContext *, void *outdata, int *outdata_size,<br />
uint8_t *buf, int buf_size);<br />
int capabilities;<br />
struct AVCodec *next;<br />
void (*flush)(AVCodecContext *);<br />
const AVRational *supported_framerates; ///array of supported framerates, or NULL if any, array is terminated by {0,0}<br />
const enum PixelFormat *pix_fmts; ///array of supported pixel formats, or NULL if unknown, array is terminanted by -1<br />
} AVCodec;<br />
<br />
Here we can see that we have some elements to name the codec, what type it is (audio/video), the supported pixel formats and some function<br />
pointers for init/encode/decode and close. Now lets see how it is used.<br />
<br />
=== libavcodec/cook.c ===<br />
If we look in this file at the bottom we can see this code:<br />
<br />
AVCodec cook_decoder =<br />
{<br />
.name = "cook",<br />
.type = CODEC_TYPE_AUDIO,<br />
.id = CODEC_ID_COOK,<br />
.priv_data_size = sizeof(COOKContext),<br />
.init = cook_decode_init,<br />
.close = cook_decode_close,<br />
.decode = cook_decode_frame,<br />
};<br />
<br />
First we get an AVCodec struct named cook_decoder. And then we set the variables of cook_decoder. Note that we only set the variables that are needed. Currently there is no encoder so we don't set any. If we now look at the id variable we can see that CODEC_ID_COOK isn't defined in libavcodec/cook.c. It is declared in avcodec.h.<br />
<br />
=== libavcodec/avcodec.h ===<br />
<br />
Here we will find the CodecID enumeration.<br />
<br />
enum CodecID {<br />
...<br />
CODEC_ID_GSM,<br />
CODEC_ID_QDM2,<br />
CODEC_ID_COOK,<br />
CODEC_ID_TRUESPEECH,<br />
CODEC_ID_TTA,<br />
...<br />
};<br />
<br />
CODEC_ID_COOK is there in the list. This is the list of all supported codecs in FFmpeg, the list is fixed and used internally to id every codec. Changing the order would break binary compatibility.<br />
<br />
This is all enough to declare a codec. Now we must register them for internal use also. This is done at runtime.<br />
<br />
=== libavcodec/allcodecs.c ===<br />
In this file we have the avcodec_register_all() function, it has entries like this for all codecs.<br />
<br />
...<br />
REGISTER_DECODER(COOK, cook);<br />
...<br />
<br />
This macro expands to a register_avcodec() call which registers a codec for internal use.<br />
Note that register_avcodec() will only be called when CONFIG_COOK_DECODER is defined.<br />
This allows to not compile the decoder code for a specific codec.<br />
But where is it defined? This is extracted by configure with this command line:<br />
<br />
sed -n 's/^[^#]*DEC.*, *\(.*\)).*/\1_decoder/p' libavcodec/allcodecs.c<br />
<br />
So adding a REGISTER_DECODER(NEW, new) entry in allcodecs.c and reconfigure is enough to add the needed define.<br />
One more thing to note is that cook.c isn't included in allcodecs.c so the symbol cook_decoder can't be<br />
found. Thus it has to be declared somewhere, that happens in <code>avcodec.h</code> and it is declared as an external symbol.<br />
extern AVCodec cook_decoder;<br />
<br />
Now we have everything to hookup a codec now we will see how the codec. For this we look into libavformat.<br />
<br />
== FFmpeg demuxer connection ==<br />
<br />
: ''[[FFmpeg demuxer howto]]''<br />
<br />
=== libavformat/rm.c ===<br />
<br />
If we think of an imaginary rm file that ffmpeg is about to process, the first thing that happens is that it is identified<br />
as a rm file. It is passed on to the rm demuxer ([http://svn.mplayerhq.hu/ffmpeg/trunk/libavformat/rm.c?view=markup rm.c]). The rm demuxer looks through the file and finds out that it is a<br />
cook file.<br />
<br />
...<br />
} else if (!strcmp(buf, "cook")) {<br />
st->codec->codec_id = CODEC_ID_COOK;<br />
...<br />
<br />
Now ffmpeg knows what codec to init and where to send the payload from the container. So back to cook.c and the initialization process.<br />
<br />
== codec code ==<br />
<br />
=== libavcodec/cook.c Init ===<br />
After ffmpeg knows what codec to use, it calls the declared initialization function pointer declared in the codecs AVCodec struct. In<br />
cook.c it is called cook_decode_init. Here we setup as much as we can before we start decoding. The following things should be handled in the init, vlc table initialization, table generation, memory allocation and extradata parsing.<br />
<br />
=== libavcodec/cook.c Close ===<br />
The cook_decode_close function is the codec clean-up call. All memory, vlc tables, etc. should be freed here.<br />
<br />
=== libavcodec/cook.c Decode ===<br />
In cook.c the name of the decode call is cook_decode_frame.<br />
<br />
static int cook_decode_frame(AVCodecContext *avctx,<br />
void *data, int *data_size,<br />
uint8_t *buf, int buf_size) {<br />
...<br />
<br />
The function has 5 arguments:<br />
* avctx is a pointer to a AVCodecContext<br />
* data is the pointer to the outbuffer<br />
* data_size is a variable that should be set to the outbuffer size in bytes<br />
* buf is the pointer to the inbuffer<br />
* buf_size is the size of the inbuffer<br />
<br />
The decode function shall return the amount of bytes consumed from the inbuffer.<br />
<br />
<br />
That's how it works without too much detail.<br />
<br />
<br />
=== The Glue codec template ===<br />
The imaginary Glue audio codec will serve as a base to exhibit bitstream reading, vlc decoding and other things.<br />
The code is purely fictional and is sometimes written purely for the sake of example. No attempt is made to prevent invalid<br />
data manipulation.<br />
<br />
The Glue codec follows.<br />
<br />
[http://wiki.multimedia.cx/index.php?title=FFmpeg_codec_howto&oldid=7347 non-colored version]<br />
<br />
<span style="font-style: italic;color: #808080;">/* First we include some default includes */</span><br />
<span style="color: #008000;">#include &lt;math.h&gt;</span><br />
<span style="color: #008000;">#include &lt;stddef.h&gt;</span><br />
<span style="color: #008000;">#include &lt;stdio.h&gt;</span><br />
<br />
<span style="font-style: italic;color: #808080;">/* The following includes have the bitstream reader, various dsp functions and the various defaults */</span><br />
<span style="color: #008000;">#define ALT_BITSTREAM_READER</span><br />
<span style="color: #008000;">#include "avcodec.h"</span><br />
<span style="color: #008000;">#include "bitstream.h"</span><br />
<span style="color: #008000;">#include "dsputil.h"</span><br />
<br />
<span style="font-style: italic;color: #808080;">/* This includes the tables needed for the Glue codec template */</span><br />
<span style="color: #008000;">#include "gluedata.h"</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* Here we declare the struct used for the codec private data */</span><br />
<span style="font-weight: bold;color: #000000;">typedef</span><span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">struct</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> GetBitContext gb;</span><br />
<span style="color: #000000;"> FFTContext fft_ctx;</span><br />
<span style="color: #000000;"> VLC vlc_table;</span><br />
<span style="color: #000000;"> MDCTContext mdct_ctx;</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">float</span><span style="color: #000000;">* sample_buffer;</span><br />
<span style="color: #000000;">} GLUEContext;</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* The init function */</span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_init(AVCodecContext *avctx)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* This imaginary codec uses one fft, one mdct and one vlc table. */</span><br />
<span style="color: #000000;"> ff_mdct_init(&amp;q-&gt;mdct_ctx, </span><span style="color: #0000ff;">10</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// 2^10 == size of mdct, 1 == inverse mdct</span><br />
<span style="color: #000000;"> ff_fft_init(&amp;q-&gt;fft_ctx, </span><span style="color: #0000ff;">9</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// 2^9 == size of fft, 0 == inverse fft</span><br />
<span style="color: #000000;"> init_vlc (&amp;q-&gt;vlc_table, </span><span style="color: #0000ff;">9</span><span style="color: #000000;">, </span><span style="color: #0000ff;">24</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> vlctable_huffbits, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> vlctable_huffcodes, </span><span style="color: #0000ff;">2</span><span style="color: #000000;">, </span><span style="color: #0000ff;">2</span><span style="color: #000000;">, </span><span style="color: #0000ff;">0</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// look in bitstream.h for the meaning of the arguments</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* We also need to allocate a sample buffer */</span><br />
<span style="color: #000000;"> q-&gt;sample_buffer = av_mallocz(</span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(</span><span style="color: #800000;">float</span><span style="color: #000000;">)*</span><span style="color: #0000ff;">1024</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// here we used av_mallocz instead of av_malloc</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">// av_mallocz memsets the whole buffer to 0</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Check if the allocation was successful */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;">(q-&gt;sample_buffer == NULL)</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> -</span><span style="color: #0000ff;">1</span><span style="color: #000000;">;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* return 0 for a successful init, -1 for failure */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> </span><span style="color: #0000ff;">0</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* This is the main decode function */</span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_frame(AVCodecContext *avctx,</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">void</span><span style="color: #000000;"> *data, </span><span style="color: #800000;">int</span><span style="color: #000000;"> *data_size,</span><br />
<span style="color: #000000;"> uint8_t *buf, </span><span style="color: #800000;">int</span><span style="color: #000000;"> buf_size)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<span style="color: #000000;"> int16_t *outbuffer = data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* We know what the arguments for this function are from above</span><br />
<span style="font-style: italic;color: #808080;"> now we just have to decode this imaginary codec, the made up</span><br />
<span style="font-style: italic;color: #808080;"> bitstream format is as follows:</span><br />
<span style="font-style: italic;color: #808080;"> 12 bits representing the amount of samples</span><br />
<span style="font-style: italic;color: #808080;"> 1 bit fft or mdct coded coeffs, 0 for fft/1 for mdct</span><br />
<span style="font-style: italic;color: #808080;"> read 13 bits representing the amount of vlc coded fft data coeffs</span><br />
<span style="font-style: italic;color: #808080;"> read 10 bits representing the amount of vlc coded mdct data coeffs</span><br />
<span style="font-style: italic;color: #808080;"> (...bits representing the coeffs...)</span><br />
<span style="font-style: italic;color: #808080;"> 5 bits of dummy data that should be ignored</span><br />
<span style="font-style: italic;color: #808080;"> 32 bits the hex value 0x12345678, used for integrity check</span><br />
<span style="font-style: italic;color: #808080;"> */</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Declare the needed variables */</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> samples, coeffs, i, fft;</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">float</span><span style="color: #000000;"> mdct_tmp[</span><span style="color: #0000ff;">1024</span><span style="color: #000000;">];</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we init the bitstream reader, we start at the beginning of the inbuffer */</span><br />
<span style="color: #000000;"> init_get_bits(&amp;q-&gt;gb, buf, buf_size*</span><span style="color: #0000ff;">8</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">//the buf_size is in bytes but we need bits</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we take 12 bits to get the amount of samples the current frame has */</span><br />
<span style="color: #000000;"> samples = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">12</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> </span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we check if we have fft or mdct coeffs */</span><br />
<span style="color: #000000;"> fft = get_bits1(&amp;q-&gt;gb);</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (fft) {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//fft coeffs, get how many</span><br />
<span style="color: #000000;"> coeffs = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">13</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> } </span><span style="font-weight: bold;color: #000000;">else</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//mdct coeffs, get how many</span><br />
<span style="color: #000000;"> coeffs = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">10</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now decode the vlc coded coeffs to the sample_buffer */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">for</span><span style="color: #000000;"> (i=</span><span style="color: #0000ff;">0</span><span style="color: #000000;"> ; i&lt;coeffs ; i++)</span><br />
<span style="color: #000000;"> q-&gt;sample_buffer[i] = get_vlc2(&amp;q-&gt;gb, q-&gt;vlc_table.table, vlc_table.bits, </span><span style="color: #0000ff;">3</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">//read about the arguments in bitstream.h</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we need to transform the coeffs to samples */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (fft) {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//The fft is done inplace</span><br />
<span style="color: #000000;"> ff_fft_permute(&amp;q-&gt;fft_ctx, (FFTComplex *) q-&gt;sample_buffer);</span><br />
<span style="color: #000000;"> ff_fft_calc(&amp;q-&gt;fft_ctx, (FFTComplex *) q-&gt;sample_buffer);</span><br />
<span style="color: #000000;"> } </span><span style="font-weight: bold;color: #000000;">else</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//And we pretend that the mdct is also inplace</span><br />
<span style="color: #000000;"> ff_imdct_calc(&amp;q-&gt;mdct_ctx, q-&gt;sample_buffer, q-&gt;sample_buffer, mdct_tmp);</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* To make it easy the stream can only be 16 bits mono, so let's convert it to that */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">for</span><span style="color: #000000;"> (i=</span><span style="color: #0000ff;">0</span><span style="color: #000000;"> ; i&lt;samples ; i++)</span><br />
<span style="color: #000000;"> outbuffer[i] = (int16_t)q-&gt;sample_buffer[i];</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Report how many samples we got */</span><br />
<span style="color: #000000;"> *data_size = samples;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Skip the dummy data bits */</span><br />
<span style="color: #000000;"> skip_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">5</span><span style="color: #000000;">);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Check if the buffer was consumed ok */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (get_bits(&amp;q-&gt;gb,</span><span style="color: #0000ff;">32</span><span style="color: #000000;">) != </span><span style="color: #008080;">0x12345678</span><span style="color: #000000;">) {</span><br />
<span style="color: #000000;"> av_log(avctx,AV_LOG_ERROR,</span><span style="color: #dd0000;">"Stream error, integrity check failed!</span><span style="color: #ff00ff;">\n</span><span style="color: #dd0000;">"</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> -</span><span style="color: #0000ff;">1</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Return the amount of bytes consumed if everything was ok */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> *data_size*</span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(int16_t);</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* the uninit function, here we just do the inverse of the init */</span><span style="color: #000000;"> </span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_close(AVCodecContext *avctx)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free allocated memory buffer */</span><br />
<span style="color: #000000;"> av_free(q-&gt;sample_buffer);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the fft transform */</span><br />
<span style="color: #000000;"> ff_fft_end(&amp;q-&gt;fft_ctx);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the mdct transform */</span><br />
<span style="color: #000000;"> ff_mdct_end(&amp;q-&gt;mdct_ctx);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the vlc table */</span><br />
<span style="color: #000000;"> free_vlc(&amp;q-&gt;vlc_table);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Return 0 if everything is ok, -1 if not */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> </span><span style="color: #0000ff;">0</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="color: #000000;">AVCodec glue_decoder =</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> .name = </span><span style="color: #dd0000;">"glue"</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> .type = CODEC_TYPE_AUDIO,</span><br />
<span style="color: #000000;"> .id = CODEC_ID_GLUE,</span><br />
<span style="color: #000000;"> .priv_data_size = </span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(GLUEContext),</span><br />
<span style="color: #000000;"> .init = glue_decode_init,</span><br />
<span style="color: #000000;"> .close = glue_decode_close,</span><br />
<span style="color: #000000;"> .decode = glue_decode_frame,</span><br />
<span style="color: #000000;">};</span><br />
<br />
== see also ==<br />
<br />
* [[FFmpeg demuxer howto]]<br />
* [[FFmpeg programming conventions]]</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=Presentation_timestamp&diff=7586Presentation timestamp2007-04-07T19:19:14Z<p>Gatoatigrado: </p>
<hr />
<div>The presentation timestamp, abbreviated pts, is the time which a frame should be displayed on screen. It starts at zero, and is multiplied by a numerator and denominator set during codec initialization. It should be set in the read_packet function in a demuxer.<br />
<br />
== see also ==<br />
<br />
* [http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-March/025999.html mailman message]</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_codec_HOWTO&diff=7585FFmpeg codec HOWTO2007-04-07T19:14:26Z<p>Gatoatigrado: </p>
<hr />
<div>This page is meant as an introduction to the internal codec API in [[FFmpeg]].<br />
It will also show how the codecs are connected with the demuxers. This is by<br />
no means a complete guide but enough to understand how to add a codec to FFmpeg.<br />
[[RealAudio cook|Cook]] is used as an example throughout.<br />
<br />
== registering the codec ==<br />
<br />
=== libavcodec/avcodec.h ===<br />
The first thing to look at is the AVCodec struct.<br />
<br />
typedef struct AVCodec {<br />
const char *name;<br />
enum CodecType type;<br />
enum CodecID id;<br />
int priv_data_size;<br />
int (*init)(AVCodecContext *);<br />
int (*encode)(AVCodecContext *, uint8_t *buf, int buf_size, void *data);<br />
int (*close)(AVCodecContext *);<br />
int (*decode)(AVCodecContext *, void *outdata, int *outdata_size,<br />
uint8_t *buf, int buf_size);<br />
int capabilities;<br />
struct AVCodec *next;<br />
void (*flush)(AVCodecContext *);<br />
const AVRational *supported_framerates; ///array of supported framerates, or NULL if any, array is terminated by {0,0}<br />
const enum PixelFormat *pix_fmts; ///array of supported pixel formats, or NULL if unknown, array is terminanted by -1<br />
} AVCodec;<br />
<br />
Here we can see that we have some elements to name the codec, what type it is (audio/video), the supported pixel formats and some function<br />
pointers for init/encode/decode and close. Now lets see how it is used.<br />
<br />
=== libavcodec/cook.c ===<br />
If we look in this file at the bottom we can see this code:<br />
<br />
AVCodec cook_decoder =<br />
{<br />
.name = "cook",<br />
.type = CODEC_TYPE_AUDIO,<br />
.id = CODEC_ID_COOK,<br />
.priv_data_size = sizeof(COOKContext),<br />
.init = cook_decode_init,<br />
.close = cook_decode_close,<br />
.decode = cook_decode_frame,<br />
};<br />
<br />
First we get an AVCodec struct named cook_decoder. And then we set the variables of cook_decoder. Note that we only set the variables that are needed. Currently there is no encoder so we don't set any. If we now look at the id variable we can see that CODEC_ID_COOK isn't defined in libavcodec/cook.c. It is declared in avcodec.h.<br />
<br />
=== libavcodec/avcodec.h ===<br />
<br />
Here we will find the CodecID enumeration.<br />
<br />
enum CodecID {<br />
...<br />
CODEC_ID_GSM,<br />
CODEC_ID_QDM2,<br />
CODEC_ID_COOK,<br />
CODEC_ID_TRUESPEECH,<br />
CODEC_ID_TTA,<br />
...<br />
};<br />
<br />
CODEC_ID_COOK is there in the list. This is the list of all supported codecs in FFmpeg, the list is fixed and used internally to id every codec. Changing the order would break binary compatibility.<br />
<br />
This is all enough to declare a codec. Now we must register them for internal use also. This is done at runtime.<br />
<br />
=== libavcodec/allcodecs.c ===<br />
In this file we have the avcodec_register_all() function, it has entries like this for all codecs.<br />
<br />
...<br />
REGISTER_DECODER(COOK, cook);<br />
...<br />
<br />
This macro expands to a register_avcodec() call which registers a codec for internal use.<br />
Note that register_avcodec() will only be called when CONFIG_COOK_DECODER is defined.<br />
This allows to not compile the decoder code for a specific codec.<br />
But where is it defined? This is extracted by configure with this command line:<br />
<br />
sed -n 's/^[^#]*DEC.*, *\(.*\)).*/\1_decoder/p' libavcodec/allcodecs.c<br />
<br />
So adding a REGISTER_DECODER(NEW, new) entry in allcodecs.c and reconfigure is enough to add the needed define.<br />
One more thing to note is that cook.c isn't included in allcodecs.c so the symbol cook_decoder can't be<br />
found. Thus it has to be declared somewhere, that happens in <code>avcodec.h</code> and it is declared as an external symbol.<br />
extern AVCodec cook_decoder;<br />
<br />
Now we have everything to hookup a codec now we will see how the codec. For this we look into libavformat.<br />
<br />
== FFmpeg demuxer connection ==<br />
<br />
: ''FFmpeg demuxer howto''<br />
<br />
=== libavformat/rm.c ===<br />
<br />
If we think of an imaginary rm file that ffmpeg is about to process, the first thing that happens is that it is identified<br />
as a rm file. It is passed on to the rm demuxer ([http://svn.mplayerhq.hu/ffmpeg/trunk/libavformat/rm.c?view=markup rm.c]). The rm demuxer looks through the file and finds out that it is a<br />
cook file.<br />
<br />
...<br />
} else if (!strcmp(buf, "cook")) {<br />
st->codec->codec_id = CODEC_ID_COOK;<br />
...<br />
<br />
Now ffmpeg knows what codec to init and where to send the payload from the container. So back to cook.c and the initialization process.<br />
<br />
== codec code ==<br />
<br />
=== libavcodec/cook.c Init ===<br />
After ffmpeg knows what codec to use, it calls the declared initialization function pointer declared in the codecs AVCodec struct. In<br />
cook.c it is called cook_decode_init. Here we setup as much as we can before we start decoding. The following things should be handled in the init, vlc table initialization, table generation, memory allocation and extradata parsing.<br />
<br />
=== libavcodec/cook.c Close ===<br />
The cook_decode_close function is the codec clean-up call. All memory, vlc tables, etc. should be freed here.<br />
<br />
=== libavcodec/cook.c Decode ===<br />
In cook.c the name of the decode call is cook_decode_frame.<br />
<br />
static int cook_decode_frame(AVCodecContext *avctx,<br />
void *data, int *data_size,<br />
uint8_t *buf, int buf_size) {<br />
...<br />
<br />
The function has 5 arguments:<br />
* avctx is a pointer to a AVCodecContext<br />
* data is the pointer to the outbuffer<br />
* data_size is a variable that should be set to the outbuffer size in bytes<br />
* buf is the pointer to the inbuffer<br />
* buf_size is the size of the inbuffer<br />
<br />
The decode function shall return the amount of bytes consumed from the inbuffer.<br />
<br />
<br />
That's how it works without too much detail.<br />
<br />
<br />
=== The Glue codec template ===<br />
The imaginary Glue audio codec will serve as a base to exhibit bitstream reading, vlc decoding and other things.<br />
The code is purely fictional and is sometimes written purely for the sake of example. No attempt is made to prevent invalid<br />
data manipulation.<br />
<br />
The Glue codec follows.<br />
<br />
[http://wiki.multimedia.cx/index.php?title=FFmpeg_codec_howto&oldid=7347 non-colored version]<br />
<br />
<span style="font-style: italic;color: #808080;">/* First we include some default includes */</span><br />
<span style="color: #008000;">#include &lt;math.h&gt;</span><br />
<span style="color: #008000;">#include &lt;stddef.h&gt;</span><br />
<span style="color: #008000;">#include &lt;stdio.h&gt;</span><br />
<br />
<span style="font-style: italic;color: #808080;">/* The following includes have the bitstream reader, various dsp functions and the various defaults */</span><br />
<span style="color: #008000;">#define ALT_BITSTREAM_READER</span><br />
<span style="color: #008000;">#include "avcodec.h"</span><br />
<span style="color: #008000;">#include "bitstream.h"</span><br />
<span style="color: #008000;">#include "dsputil.h"</span><br />
<br />
<span style="font-style: italic;color: #808080;">/* This includes the tables needed for the Glue codec template */</span><br />
<span style="color: #008000;">#include "gluedata.h"</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* Here we declare the struct used for the codec private data */</span><br />
<span style="font-weight: bold;color: #000000;">typedef</span><span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">struct</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> GetBitContext gb;</span><br />
<span style="color: #000000;"> FFTContext fft_ctx;</span><br />
<span style="color: #000000;"> VLC vlc_table;</span><br />
<span style="color: #000000;"> MDCTContext mdct_ctx;</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">float</span><span style="color: #000000;">* sample_buffer;</span><br />
<span style="color: #000000;">} GLUEContext;</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* The init function */</span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_init(AVCodecContext *avctx)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* This imaginary codec uses one fft, one mdct and one vlc table. */</span><br />
<span style="color: #000000;"> ff_mdct_init(&amp;q-&gt;mdct_ctx, </span><span style="color: #0000ff;">10</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// 2^10 == size of mdct, 1 == inverse mdct</span><br />
<span style="color: #000000;"> ff_fft_init(&amp;q-&gt;fft_ctx, </span><span style="color: #0000ff;">9</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// 2^9 == size of fft, 0 == inverse fft</span><br />
<span style="color: #000000;"> init_vlc (&amp;q-&gt;vlc_table, </span><span style="color: #0000ff;">9</span><span style="color: #000000;">, </span><span style="color: #0000ff;">24</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> vlctable_huffbits, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> vlctable_huffcodes, </span><span style="color: #0000ff;">2</span><span style="color: #000000;">, </span><span style="color: #0000ff;">2</span><span style="color: #000000;">, </span><span style="color: #0000ff;">0</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// look in bitstream.h for the meaning of the arguments</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* We also need to allocate a sample buffer */</span><br />
<span style="color: #000000;"> q-&gt;sample_buffer = av_mallocz(</span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(</span><span style="color: #800000;">float</span><span style="color: #000000;">)*</span><span style="color: #0000ff;">1024</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// here we used av_mallocz instead of av_malloc</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">// av_mallocz memsets the whole buffer to 0</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Check if the allocation was successful */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;">(q-&gt;sample_buffer == NULL)</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> -</span><span style="color: #0000ff;">1</span><span style="color: #000000;">;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* return 0 for a successful init, -1 for failure */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> </span><span style="color: #0000ff;">0</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* This is the main decode function */</span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_frame(AVCodecContext *avctx,</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">void</span><span style="color: #000000;"> *data, </span><span style="color: #800000;">int</span><span style="color: #000000;"> *data_size,</span><br />
<span style="color: #000000;"> uint8_t *buf, </span><span style="color: #800000;">int</span><span style="color: #000000;"> buf_size)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<span style="color: #000000;"> int16_t *outbuffer = data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* We know what the arguments for this function are from above</span><br />
<span style="font-style: italic;color: #808080;"> now we just have to decode this imaginary codec, the made up</span><br />
<span style="font-style: italic;color: #808080;"> bitstream format is as follows:</span><br />
<span style="font-style: italic;color: #808080;"> 12 bits representing the amount of samples</span><br />
<span style="font-style: italic;color: #808080;"> 1 bit fft or mdct coded coeffs, 0 for fft/1 for mdct</span><br />
<span style="font-style: italic;color: #808080;"> read 13 bits representing the amount of vlc coded fft data coeffs</span><br />
<span style="font-style: italic;color: #808080;"> read 10 bits representing the amount of vlc coded mdct data coeffs</span><br />
<span style="font-style: italic;color: #808080;"> (...bits representing the coeffs...)</span><br />
<span style="font-style: italic;color: #808080;"> 5 bits of dummy data that should be ignored</span><br />
<span style="font-style: italic;color: #808080;"> 32 bits the hex value 0x12345678, used for integrity check</span><br />
<span style="font-style: italic;color: #808080;"> */</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Declare the needed variables */</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> samples, coeffs, i, fft;</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">float</span><span style="color: #000000;"> mdct_tmp[</span><span style="color: #0000ff;">1024</span><span style="color: #000000;">];</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we init the bitstream reader, we start at the beginning of the inbuffer */</span><br />
<span style="color: #000000;"> init_get_bits(&amp;q-&gt;gb, buf, buf_size*</span><span style="color: #0000ff;">8</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">//the buf_size is in bytes but we need bits</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we take 12 bits to get the amount of samples the current frame has */</span><br />
<span style="color: #000000;"> samples = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">12</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> </span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we check if we have fft or mdct coeffs */</span><br />
<span style="color: #000000;"> fft = get_bits1(&amp;q-&gt;gb);</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (fft) {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//fft coeffs, get how many</span><br />
<span style="color: #000000;"> coeffs = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">13</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> } </span><span style="font-weight: bold;color: #000000;">else</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//mdct coeffs, get how many</span><br />
<span style="color: #000000;"> coeffs = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">10</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now decode the vlc coded coeffs to the sample_buffer */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">for</span><span style="color: #000000;"> (i=</span><span style="color: #0000ff;">0</span><span style="color: #000000;"> ; i&lt;coeffs ; i++)</span><br />
<span style="color: #000000;"> q-&gt;sample_buffer[i] = get_vlc2(&amp;q-&gt;gb, q-&gt;vlc_table.table, vlc_table.bits, </span><span style="color: #0000ff;">3</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">//read about the arguments in bitstream.h</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we need to transform the coeffs to samples */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (fft) {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//The fft is done inplace</span><br />
<span style="color: #000000;"> ff_fft_permute(&amp;q-&gt;fft_ctx, (FFTComplex *) q-&gt;sample_buffer);</span><br />
<span style="color: #000000;"> ff_fft_calc(&amp;q-&gt;fft_ctx, (FFTComplex *) q-&gt;sample_buffer);</span><br />
<span style="color: #000000;"> } </span><span style="font-weight: bold;color: #000000;">else</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//And we pretend that the mdct is also inplace</span><br />
<span style="color: #000000;"> ff_imdct_calc(&amp;q-&gt;mdct_ctx, q-&gt;sample_buffer, q-&gt;sample_buffer, mdct_tmp);</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* To make it easy the stream can only be 16 bits mono, so let's convert it to that */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">for</span><span style="color: #000000;"> (i=</span><span style="color: #0000ff;">0</span><span style="color: #000000;"> ; i&lt;samples ; i++)</span><br />
<span style="color: #000000;"> outbuffer[i] = (int16_t)q-&gt;sample_buffer[i];</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Report how many samples we got */</span><br />
<span style="color: #000000;"> *data_size = samples;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Skip the dummy data bits */</span><br />
<span style="color: #000000;"> skip_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">5</span><span style="color: #000000;">);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Check if the buffer was consumed ok */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (get_bits(&amp;q-&gt;gb,</span><span style="color: #0000ff;">32</span><span style="color: #000000;">) != </span><span style="color: #008080;">0x12345678</span><span style="color: #000000;">) {</span><br />
<span style="color: #000000;"> av_log(avctx,AV_LOG_ERROR,</span><span style="color: #dd0000;">"Stream error, integrity check failed!</span><span style="color: #ff00ff;">\n</span><span style="color: #dd0000;">"</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> -</span><span style="color: #0000ff;">1</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Return the amount of bytes consumed if everything was ok */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> *data_size*</span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(int16_t);</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* the uninit function, here we just do the inverse of the init */</span><span style="color: #000000;"> </span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_close(AVCodecContext *avctx)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free allocated memory buffer */</span><br />
<span style="color: #000000;"> av_free(q-&gt;sample_buffer);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the fft transform */</span><br />
<span style="color: #000000;"> ff_fft_end(&amp;q-&gt;fft_ctx);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the mdct transform */</span><br />
<span style="color: #000000;"> ff_mdct_end(&amp;q-&gt;mdct_ctx);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the vlc table */</span><br />
<span style="color: #000000;"> free_vlc(&amp;q-&gt;vlc_table);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Return 0 if everything is ok, -1 if not */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> </span><span style="color: #0000ff;">0</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="color: #000000;">AVCodec glue_decoder =</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> .name = </span><span style="color: #dd0000;">"glue"</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> .type = CODEC_TYPE_AUDIO,</span><br />
<span style="color: #000000;"> .id = CODEC_ID_GLUE,</span><br />
<span style="color: #000000;"> .priv_data_size = </span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(GLUEContext),</span><br />
<span style="color: #000000;"> .init = glue_decode_init,</span><br />
<span style="color: #000000;"> .close = glue_decode_close,</span><br />
<span style="color: #000000;"> .decode = glue_decode_frame,</span><br />
<span style="color: #000000;">};</span><br />
<br />
== see also ==<br />
<br />
* [[FFmpeg demuxer howto]]<br />
* [[FFmpeg programming conventions]]</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_programming_conventions&diff=7584FFmpeg programming conventions2007-04-07T19:10:58Z<p>Gatoatigrado: </p>
<hr />
<div>: ''please make this page more complete if you can, thanks''<br />
<br />
== Memory allocation ==<br />
<br />
: av_malloc / av_free / av_realloc - correspond to malloc functions, currently only used as wrappers<br />
: av_fast_realloc - a shortcut to common buffer reallocation functions<br />
<br />
== Debug printing ==<br />
<br />
: av_log(avctx, print_type, [same argument style as printf])<br />
some options for print_type are AV_LOG_VERBOSE / AV_LOG_ERROR.<br />
: dprintf(avctx, [same argument style as printf]) is an alias for AV_LOG_DEBUG<br />
<br />
== Stream and array operations ==<br />
<br />
: AV_RB / AV_RL - read little endian or big endian from an array. e.g. int result = AV_RB32(&array[offset])<br />
<br />
functions that automatically increment arguments<br />
: bytestream_get_buffer(&buffer, destination, length) - copies from buffer. buffer type is uint8_t **.<br />
: bytestream_get_le16/24/32(&buffer) - get an int.<br />
<br />
: get_byte(pb) - get a byte, where pb is the ByteIOContext in a demuxer (typically), in (AVFormatContext) &s->pb<br />
: get_be/get_le 16/24/32(pb) - get an integer<br />
: get_buffer(pb, buffer, length) - get an array, returns number of bytes read. typically, return if number of bytes read != length<br />
<br />
== see also ==<br />
<br />
* [[FFmpeg codec howto]]<br />
* [[FFmpeg demuxer howto]]</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_demuxer_HOWTO&diff=7583FFmpeg demuxer HOWTO2007-04-07T18:53:53Z<p>Gatoatigrado: </p>
<hr />
<div>: ''please make this page more complete if you can, thanks''<br />
<br />
== registering the demuxer ==<br />
<br />
=== libavformat/allformats.h ===<br />
<br />
Add "<code>extern AVInputFormat demuxername_demuxer</code>" or "<code>extern AVOutputFormat demuxername</code>" in the correct section, in alphabetical order.<br />
<br />
=== libavformat/allformats.c ===<br />
<br />
Add "<code>REGISTER_DEMUXER(DEMUXERNAME, demuxername)</code>". The macro will add the _demuxer or whatever is appropriate. '''Run configure in the top level directory and make clean, as config.h will be automatically modified'''.<br />
<br />
=== libavformat/Makefile ===<br />
<br />
Add OBJS-$(CONFIG_DEMUXERNAME_DEMUXER) += filename.o<br />
<br />
== demuxer code ==<br />
<br />
This section is merely an overview; please look at an actual demuxer to see how to do it.<br />
<br />
=== demuxername_probe ===<br />
<br />
The probe will be called by the framework; this is where you can detect if the file is your format. There should be no dependence on filename, in fact you probably can't even access it. Most video containers start with some constant sequence of bytes. If there are 4 bytes and they are always the same, and unique, then return AVPROBE_SCORE_MAX. Otherwise, return 0. If the format is mis-designed and doesn't have this, divide AVPROBE_SCORE_MAX by something (e.g. 2) to designate limited confidence that the file corresponds to your demuxer.<br />
<br />
=== demuxername_read_header ===<br />
<br />
Here you should get any header information, e.g. the frame width if it is video, the sample rate if it is audio, etc. You should also setup your streams (looking at examples usually explains most things, except perhaps [[presentation timestamp|timestamp]] information); set codec_type, codec_id, and possibly width, height, pix_fmt for video, and channels, sample_rate, bits_per_sample, and bit_rate for audio. return 0.<br />
<br />
=== demuxername_read_packet ===<br />
<br />
Most formats are split into blocks of audio and video. The read_packet function should set up pkt. This involves initializing the packet with av_new_packet, or, preferably, av_get_packet (which also takes the file stream as an argument). After you initialize the packet, you should set (in any order) pos and data (if you used av_new_packet), stream_index (to set which decoder, usually audio or video, to send the packet to, which corresponds to the order you initialized them in), and pts, the presentation timestamp.<br />
<br />
== see also ==<br />
<br />
* [[FFmpeg codec howto]]<br />
* [[FFmpeg programming conventions]]</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_codec_HOWTO&diff=7508FFmpeg codec HOWTO2007-03-24T21:38:12Z<p>Gatoatigrado: /* The Glue codec template */</p>
<hr />
<div>This page is meant as an introduction to the internal codec API in [[FFmpeg]].<br />
It will also show how the codecs are connected with the demuxers. This is by<br />
no means a complete guide but enough to understand how to add a codec to FFmpeg.<br />
[[RealAudio cook|Cook]] is used as an example throughout.<br />
<br />
== libavcodec/avcodec.h ==<br />
The first thing to look at is the AVCodec struct.<br />
<br />
typedef struct AVCodec {<br />
const char *name;<br />
enum CodecType type;<br />
enum CodecID id;<br />
int priv_data_size;<br />
int (*init)(AVCodecContext *);<br />
int (*encode)(AVCodecContext *, uint8_t *buf, int buf_size, void *data);<br />
int (*close)(AVCodecContext *);<br />
int (*decode)(AVCodecContext *, void *outdata, int *outdata_size,<br />
uint8_t *buf, int buf_size);<br />
int capabilities;<br />
struct AVCodec *next;<br />
void (*flush)(AVCodecContext *);<br />
const AVRational *supported_framerates; ///array of supported framerates, or NULL if any, array is terminated by {0,0}<br />
const enum PixelFormat *pix_fmts; ///array of supported pixel formats, or NULL if unknown, array is terminanted by -1<br />
} AVCodec;<br />
<br />
Here we can see that we have some elements to name the codec, what type it is (audio/video), the supported pixel formats and some function<br />
pointers for init/encode/decode and close. Now lets see how it is used.<br />
<br />
== libavcodec/cook.c ==<br />
If we look in this file at the bottom we can see this code:<br />
<br />
AVCodec cook_decoder =<br />
{<br />
.name = "cook",<br />
.type = CODEC_TYPE_AUDIO,<br />
.id = CODEC_ID_COOK,<br />
.priv_data_size = sizeof(COOKContext),<br />
.init = cook_decode_init,<br />
.close = cook_decode_close,<br />
.decode = cook_decode_frame,<br />
};<br />
<br />
First we get an AVCodec struct named cook_decoder. And then we set the variables of cook_decoder. Note that we only set the variables that are needed. Currently there is no encoder so we don't set any. If we now look at the id variable we can see that CODEC_ID_COOK isn't defined in libavcodec/cook.c. It is declared in avcodec.h.<br />
<br />
== libavcodec/avcodec.h ==<br />
<br />
Here we will find the CodecID enumeration.<br />
<br />
enum CodecID {<br />
...<br />
CODEC_ID_GSM,<br />
CODEC_ID_QDM2,<br />
CODEC_ID_COOK,<br />
CODEC_ID_TRUESPEECH,<br />
CODEC_ID_TTA,<br />
...<br />
};<br />
<br />
CODEC_ID_COOK is there in the list. This is the list of all supported codecs in FFmpeg, the list is fixed and used internally to id every codec. Changing the order would break binary compatibility.<br />
<br />
This is all enough to declare a codec. Now we must register them for internal use also. This is done at runtime.<br />
<br />
== libavcodec/allcodecs.c ==<br />
In this file we have the avcodec_register_all() function, it has entries like this for all codecs.<br />
<br />
...<br />
REGISTER_DECODER(COOK, cook);<br />
...<br />
<br />
This macro expands to a register_avcodec() call which registers a codec for internal use.<br />
Note that register_avcodec() will only be called when CONFIG_COOK_DECODER is defined.<br />
This allows to not compile the decoder code for a specific codec.<br />
But where is it defined? This is extracted by configure with this command line:<br />
<br />
sed -n 's/^[^#]*DEC.*, *\(.*\)).*/\1_decoder/p' libavcodec/allcodecs.c<br />
<br />
So adding a REGISTER_DECODER(NEW, new) entry in allcodecs.c and reconfigure is enough to add the needed define.<br />
One more thing to note is that cook.c isn't included in allcodecs.c so the symbol cook_decoder can't be<br />
found. Thus it has to be declared somewhere, that happens in <code>avcodec.h</code> and it is declared as an external symbol.<br />
extern AVCodec cook_decoder;<br />
<br />
Now we have everything to hookup a codec now we will see how the codec. For this we look into libavformat.<br />
<br />
== libavformat/rm.c ==<br />
If we think of an imaginary rm file that ffmpeg is about to process, the first thing that happens is that it is identified<br />
as a rm file. It is passed on to the rm demuxer (rm.c). The rm demuxer looks through the file and finds out that it is a<br />
cook file.<br />
<br />
...<br />
} else if (!strcmp(buf, "cook")) {<br />
st->codec->codec_id = CODEC_ID_COOK;<br />
...<br />
<br />
Now ffmpeg knows what codec to init and where to send the payload from the container. So back to cook.c and the initialization process.<br />
<br />
== libavcodec/cook.c Init ==<br />
After ffmpeg knows what codec to use, it calls the declared initialization function pointer declared in the codecs AVCodec struct. In<br />
cook.c it is called cook_decode_init. Here we setup as much as we can before we start decoding. The following things should be handled in the init, vlc table initialization, table generation, memory allocation and extradata parsing.<br />
<br />
== libavcodec/cook.c Close ==<br />
The cook_decode_close function is the codec clean-up call. All memory, vlc tables, etc. should be freed here.<br />
<br />
== libavcodec/cook.c Decode ==<br />
In cook.c the name of the decode call is cook_decode_frame.<br />
<br />
static int cook_decode_frame(AVCodecContext *avctx,<br />
void *data, int *data_size,<br />
uint8_t *buf, int buf_size) {<br />
...<br />
<br />
The function has 5 arguments:<br />
* avctx is a pointer to a AVCodecContext<br />
* data is the pointer to the outbuffer<br />
* data_size is a variable that should be set to the outbuffer size in bytes<br />
* buf is the pointer to the inbuffer<br />
* buf_size is the size of the inbuffer<br />
<br />
The decode function shall return the amount of bytes consumed from the inbuffer.<br />
<br />
<br />
That's how it works without too much detail.<br />
<br />
<br />
== The Glue codec template ==<br />
The imaginary Glue audio codec will serve as a base to exhibit bitstream reading, vlc decoding and other things.<br />
The code is purely fictional and is sometimes written purely for the sake of example. No attempt is made to prevent invalid<br />
data manipulation.<br />
<br />
The Glue codec follows.<br />
<br />
[http://wiki.multimedia.cx/index.php?title=FFmpeg_codec_howto&oldid=7347 non-colored version]<br />
<br />
<span style="font-style: italic;color: #808080;">/* First we include some default includes */</span><br />
<span style="color: #008000;">#include &lt;math.h&gt;</span><br />
<span style="color: #008000;">#include &lt;stddef.h&gt;</span><br />
<span style="color: #008000;">#include &lt;stdio.h&gt;</span><br />
<br />
<span style="font-style: italic;color: #808080;">/* The following includes have the bitstream reader, various dsp functions and the various defaults */</span><br />
<span style="color: #008000;">#define ALT_BITSTREAM_READER</span><br />
<span style="color: #008000;">#include "avcodec.h"</span><br />
<span style="color: #008000;">#include "bitstream.h"</span><br />
<span style="color: #008000;">#include "dsputil.h"</span><br />
<br />
<span style="font-style: italic;color: #808080;">/* This includes the tables needed for the Glue codec template */</span><br />
<span style="color: #008000;">#include "gluedata.h"</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* Here we declare the struct used for the codec private data */</span><br />
<span style="font-weight: bold;color: #000000;">typedef</span><span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">struct</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> GetBitContext gb;</span><br />
<span style="color: #000000;"> FFTContext fft_ctx;</span><br />
<span style="color: #000000;"> VLC vlc_table;</span><br />
<span style="color: #000000;"> MDCTContext mdct_ctx;</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">float</span><span style="color: #000000;">* sample_buffer;</span><br />
<span style="color: #000000;">} GLUEContext;</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* The init function */</span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_init(AVCodecContext *avctx)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* This imaginary codec uses one fft, one mdct and one vlc table. */</span><br />
<span style="color: #000000;"> ff_mdct_init(&amp;q-&gt;mdct_ctx, </span><span style="color: #0000ff;">10</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// 2^10 == size of mdct, 1 == inverse mdct</span><br />
<span style="color: #000000;"> ff_fft_init(&amp;q-&gt;fft_ctx, </span><span style="color: #0000ff;">9</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// 2^9 == size of fft, 0 == inverse fft</span><br />
<span style="color: #000000;"> init_vlc (&amp;q-&gt;vlc_table, </span><span style="color: #0000ff;">9</span><span style="color: #000000;">, </span><span style="color: #0000ff;">24</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> vlctable_huffbits, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">, </span><span style="color: #0000ff;">1</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> vlctable_huffcodes, </span><span style="color: #0000ff;">2</span><span style="color: #000000;">, </span><span style="color: #0000ff;">2</span><span style="color: #000000;">, </span><span style="color: #0000ff;">0</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// look in bitstream.h for the meaning of the arguments</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* We also need to allocate a sample buffer */</span><br />
<span style="color: #000000;"> q-&gt;sample_buffer = av_mallocz(</span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(</span><span style="color: #800000;">float</span><span style="color: #000000;">)*</span><span style="color: #0000ff;">1024</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">// here we used av_mallocz instead of av_malloc</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">// av_mallocz memsets the whole buffer to 0</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Check if the allocation was successful */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;">(q-&gt;sample_buffer == NULL)</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> -</span><span style="color: #0000ff;">1</span><span style="color: #000000;">;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* return 0 for a successful init, -1 for failure */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> </span><span style="color: #0000ff;">0</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* This is the main decode function */</span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_frame(AVCodecContext *avctx,</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">void</span><span style="color: #000000;"> *data, </span><span style="color: #800000;">int</span><span style="color: #000000;"> *data_size,</span><br />
<span style="color: #000000;"> uint8_t *buf, </span><span style="color: #800000;">int</span><span style="color: #000000;"> buf_size)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<span style="color: #000000;"> int16_t *outbuffer = data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* We know what the arguments for this function are from above</span><br />
<span style="font-style: italic;color: #808080;"> now we just have to decode this imaginary codec, the made up</span><br />
<span style="font-style: italic;color: #808080;"> bitstream format is as follows:</span><br />
<span style="font-style: italic;color: #808080;"> 12 bits representing the amount of samples</span><br />
<span style="font-style: italic;color: #808080;"> 1 bit fft or mdct coded coeffs, 0 for fft/1 for mdct</span><br />
<span style="font-style: italic;color: #808080;"> read 13 bits representing the amount of vlc coded fft data coeffs</span><br />
<span style="font-style: italic;color: #808080;"> read 10 bits representing the amount of vlc coded mdct data coeffs</span><br />
<span style="font-style: italic;color: #808080;"> (...bits representing the coeffs...)</span><br />
<span style="font-style: italic;color: #808080;"> 5 bits of dummy data that should be ignored</span><br />
<span style="font-style: italic;color: #808080;"> 32 bits the hex value 0x12345678, used for integrity check</span><br />
<span style="font-style: italic;color: #808080;"> */</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Declare the needed variables */</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> samples, coeffs, i, fft;</span><br />
<span style="color: #000000;"> </span><span style="color: #800000;">float</span><span style="color: #000000;"> mdct_tmp[</span><span style="color: #0000ff;">1024</span><span style="color: #000000;">];</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we init the bitstream reader, we start at the beginning of the inbuffer */</span><br />
<span style="color: #000000;"> init_get_bits(&amp;q-&gt;gb, buf, buf_size*</span><span style="color: #0000ff;">8</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">//the buf_size is in bytes but we need bits</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we take 12 bits to get the amount of samples the current frame has */</span><br />
<span style="color: #000000;"> samples = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">12</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> </span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we check if we have fft or mdct coeffs */</span><br />
<span style="color: #000000;"> fft = get_bits1(&amp;q-&gt;gb);</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (fft) {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//fft coeffs, get how many</span><br />
<span style="color: #000000;"> coeffs = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">13</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> } </span><span style="font-weight: bold;color: #000000;">else</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//mdct coeffs, get how many</span><br />
<span style="color: #000000;"> coeffs = get_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">10</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now decode the vlc coded coeffs to the sample_buffer */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">for</span><span style="color: #000000;"> (i=</span><span style="color: #0000ff;">0</span><span style="color: #000000;"> ; i&lt;coeffs ; i++)</span><br />
<span style="color: #000000;"> q-&gt;sample_buffer[i] = get_vlc2(&amp;q-&gt;gb, q-&gt;vlc_table.table, vlc_table.bits, </span><span style="color: #0000ff;">3</span><span style="color: #000000;">); </span><span style="font-style: italic;color: #808080;">//read about the arguments in bitstream.h</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Now we need to transform the coeffs to samples */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (fft) {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//The fft is done inplace</span><br />
<span style="color: #000000;"> ff_fft_permute(&amp;q-&gt;fft_ctx, (FFTComplex *) q-&gt;sample_buffer);</span><br />
<span style="color: #000000;"> ff_fft_calc(&amp;q-&gt;fft_ctx, (FFTComplex *) q-&gt;sample_buffer);</span><br />
<span style="color: #000000;"> } </span><span style="font-weight: bold;color: #000000;">else</span><span style="color: #000000;"> {</span><br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">//And we pretend that the mdct is also inplace</span><br />
<span style="color: #000000;"> ff_imdct_calc(&amp;q-&gt;mdct_ctx, q-&gt;sample_buffer, q-&gt;sample_buffer, mdct_tmp);</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* To make it easy the stream can only be 16 bits mono, so let's convert it to that */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">for</span><span style="color: #000000;"> (i=</span><span style="color: #0000ff;">0</span><span style="color: #000000;"> ; i&lt;samples ; i++)</span><br />
<span style="color: #000000;"> outbuffer[i] = (int16_t)q-&gt;sample_buffer[i];</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Report how many samples we got */</span><br />
<span style="color: #000000;"> *data_size = samples;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Skip the dummy data bits */</span><br />
<span style="color: #000000;"> skip_bits(&amp;q-&gt;gb, </span><span style="color: #0000ff;">5</span><span style="color: #000000;">);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Check if the buffer was consumed ok */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">if</span><span style="color: #000000;"> (get_bits(&amp;q-&gt;gb,</span><span style="color: #0000ff;">32</span><span style="color: #000000;">) != </span><span style="color: #008080;">0x12345678</span><span style="color: #000000;">) {</span><br />
<span style="color: #000000;"> av_log(avctx,AV_LOG_ERROR,</span><span style="color: #dd0000;">"Stream error, integrity check failed!</span><span style="color: #ff00ff;">\n</span><span style="color: #dd0000;">"</span><span style="color: #000000;">);</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> -</span><span style="color: #0000ff;">1</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;"> }</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Return the amount of bytes consumed if everything was ok */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> *data_size*</span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(int16_t);</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="font-style: italic;color: #808080;">/* the uninit function, here we just do the inverse of the init */</span><span style="color: #000000;"> </span><br />
<span style="color: #800000;">static</span><span style="color: #000000;"> </span><span style="color: #800000;">int</span><span style="color: #000000;"> glue_decode_close(AVCodecContext *avctx)</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> GLUEContext *q = avctx-&gt;priv_data;</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free allocated memory buffer */</span><br />
<span style="color: #000000;"> av_free(q-&gt;sample_buffer);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the fft transform */</span><br />
<span style="color: #000000;"> ff_fft_end(&amp;q-&gt;fft_ctx);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the mdct transform */</span><br />
<span style="color: #000000;"> ff_mdct_end(&amp;q-&gt;mdct_ctx);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Free the vlc table */</span><br />
<span style="color: #000000;"> free_vlc(&amp;q-&gt;vlc_table);</span><br />
<br />
<span style="color: #000000;"> </span><span style="font-style: italic;color: #808080;">/* Return 0 if everything is ok, -1 if not */</span><br />
<span style="color: #000000;"> </span><span style="font-weight: bold;color: #000000;">return</span><span style="color: #000000;"> </span><span style="color: #0000ff;">0</span><span style="color: #000000;">;</span><br />
<span style="color: #000000;">}</span><br />
<br />
<br />
<span style="color: #000000;">AVCodec glue_decoder =</span><br />
<span style="color: #000000;">{</span><br />
<span style="color: #000000;"> .name = </span><span style="color: #dd0000;">"glue"</span><span style="color: #000000;">,</span><br />
<span style="color: #000000;"> .type = CODEC_TYPE_AUDIO,</span><br />
<span style="color: #000000;"> .id = CODEC_ID_GLUE,</span><br />
<span style="color: #000000;"> .priv_data_size = </span><span style="font-weight: bold;color: #000000;">sizeof</span><span style="color: #000000;">(GLUEContext),</span><br />
<span style="color: #000000;"> .init = glue_decode_init,</span><br />
<span style="color: #000000;"> .close = glue_decode_close,</span><br />
<span style="color: #000000;"> .decode = glue_decode_frame,</span><br />
<span style="color: #000000;">};</span></div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_Of_Code_2007&diff=7506FFmpeg Summer Of Code 20072007-03-24T05:12:03Z<p>Gatoatigrado: /* Bethsoft VID */</p>
<hr />
<div>Google is sponsoring their third annual [http://code.google.com/soc/ Summer of Code] for the summer of 2007. This entails sponsoring students to work on assorted open source projects as well as sponsoring mentors in those same projects. Everyone wins.<br />
<br />
[[FFmpeg]] was a Summer of Code participant in the [http://code.google.com/soc/2006/ffmpeg/about.html summer of 2006] (here is the [[FFmpeg Summer Of Code|corresponding Wiki page]]).<br />
<br />
[[User:Multimedia Mike|Mike Melanson]] (mike -at- multimedia.cx) is the administrator and main point of contact for matters relating to the FFmpeg Summer of Code.<br />
<br />
== How to apply ==<br />
<br />
Before you can apply make sure you are qualified enough to apply. Last year 50% of the applicants weren't qualified for the task they applied for.<br />
<br />
* You have to know how to program in C fairly well.<br />
* We would like you to submit a patch that fixes a bug or adds a feature to FFmpeg. By doing that we will know that you are qualified for the task or not. On this page there is a list of [[Summer Of Code 2007#Qualification_tasks|Qualification Tasks]] that can be done. But you are free to submit anything you feel might be of value to FFmpeg. The qualification task can be done after you have filed you application (up until around April 7 as the list of accepted students is scheduled to be posted by Google on April 9).<br />
* Submit a good application through the formal Google Summer of Code process during the application timeframe (March 14-24, 2007).<br />
* You have to have >35 hours per week to put into the project.<br />
* You can't have another job at the same time as the SoC project.<br />
<br />
== Current Status ==<br />
* March 5-12, 2007: Application period for mentoring organizations.<br />
* March 6, 2007: Mike Melanson submitted FFmpeg mentoring application.<br />
* March 14, 2007: Google are now accepting student Applications.<br />
* March 15, 2007: FFmpeg got accepted as mentoring organization; now accepting student applications until March 24, 2007. Note that this is '''NOT''' the deadline to complete a qualification task; final student selections are to be made by April 9.<br />
<br />
== Project Proposals ==<br />
<br />
=== Overview ===<br />
Qualifications for a good Summer of Code proposal:<br />
* discrete, well-defined, modular<br />
* comprised of a series of measurable sub-goals<br />
* based on open specs that are available free of charge<br />
* based on complete specs<br />
An example of a good proposal is the implementation of a decoder or demuxer for an as yet unsupported multimedia format, or an encoder or muxer for a format that can already be demuxed/decoded by FFmpeg.<br />
<br />
An example of a less desirable proposal is one that's not as measurable, such as refactoring APIs. Bad proposals tend to be ones that would require touching a lot of core code.<br />
<br />
To re-iterate:<br />
* Localized/isolated code projects = ''good''<br />
* Global code refactoring = ''bad''<br />
<br />
=== Note ===<br />
THIS LIST IS NOT THE PROPOSALS WE ARE SUBMITTING!<br />
<br />
Most of this list is just some ideas we are kicking around.<br />
<br />
=== Projects with Mentors (official projects) ===<br />
<br />
==== QCELP Decoder ====<br />
<br />
* Specification: QCELP decoder [http://www.3gpp2.org/Public_html/specs/alltsgscfm.cfm spec] is c.s0020 and source is c.r0020<br />
* Sample files: http://samples.mplayerhq.hu/A-codecs/qclp/<br />
<br />
''Mentor: Benjamin Larsson''<br />
<br />
==== Matroska Muxer ====<br />
<br />
* Specification: http://www.matroska.org/technical/specs/index.html<br />
* Sample files: http://samples.mplayerhq.hu/Matroska/<br />
<br />
''Mentor: Aurelien Jacobs; Backup mentors: Steve Lhomme, Ronald S. Bultje''<br />
<br />
==== MPEG TS/DVB Muxer ====<br />
* Specification: ISO 13818-1<br />
<br />
''Mentor: Baptiste Coudurier''<br />
<br />
==== MXF Muxer ====<br />
* Specification: SMPTE 377M<br />
<br />
''Mentor: Baptiste Coudurier''<br />
<br />
==== RV40 Decoder ====<br />
* [[RealVideo 4]] is steadily being reverse engineered and should be a reasonable candidate for re-implementation by the summer.<br />
<br />
''Mentor: Mike Melanson''<br />
<br />
==== PAFF decoding for H.264/AVC ====<br />
* Specification: [http://www.itu.int/rec/T-REC-H.264-200503-I/en ITU-T]<br />
* Sample files: http://samples.mplayerhq.hu/V-codecs/h264/PAFF/<br />
<br />
''Mentor: Loren Merritt''<br />
<br />
==== Dirac Encoder AND Decoder ====<br />
* Website: http://dirac.sf.net<br />
* Specification: http://dirac.sourceforge.net/specification.html<br />
<br />
''Mentor: Luca Barbato''<br />
<br />
==== E-AC3 Decoder ====<br />
* Specification: http://www.atsc.org/standards/a_52b.pdf<br />
* Samples: http://samples.mplayerhq.hu/evob/MAININTRO.EVO<br />
<br />
''Mentor: Justin Ruggles''<br />
<br />
<br />
<br />
=== Ideas for more projects to be determined ===<br />
<br />
==== JPEG2000 ====<br />
* Specifications: [http://www.itu.int/rec/T-REC-T/e As ITU-T recommendations], [http://isotc.iso.org/livelink/livelink/fetch/2000/2489/Ittf_Home/PubliclyAvailableStandards.htm ISO publicly available standards]<br />
* Sources: [http://www.ece.uvic.ca/~mdadams/jasper/ JasPer], [http://www.openjpeg.org/ OpenJpeg], [http://jj2000.epfl.ch/ JJ2000 (in Java)]<br />
* Samples: http://samples.mplayerhq.hu/jpeg2000/<br />
<br />
==== Monkey's Audio Decoder (APE) ====<br />
* Sources: [http://www.monkeysaudio.com/files/MAC_SDK_399.zip original sources], [http://sourceforge.net/projects/mac-port/ original sources port for non-win32 platforms], [http://jmac.sourceforge.net/ LGPLed Java implementation]<br />
<br />
==== Finish LC-AAC decoder and implement HE-AAC decoder (LGPL) ====<br />
* LGPL code: http://svn.mplayerhq.hu/aac/<br />
* GPL decoder (libfaad2): http://www.audiocoding.com<br />
* Possible AAC+ sub-project:<br />
** [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] decoder (aacPlus v1 support)<br />
** [http://www.codingtechnologies.com/products/paraSter.htm PS (Parametric Stereo)] decoder (for aacPlus v2 support)<br />
<br />
==== GSM Decoder ====<br />
* Specification + sample implementation: http://kbs.cs.tu-berlin.de/~jutta/toast.html<br />
* Samples: http://samples.mplayerhq.hu/A-codecs/GSM/<br />
<br />
==== i263 Decoder ====<br />
* Specification: [[I263 | I263 Format Specification at MultimediaWiki]]<br />
* Sources: [http://multimedia.cx/I263Src.zip GPLed I263 decoder]<br />
* Sample files: http://samples.mplayerhq.hu/V-codecs/I263/<br />
<br />
==== VP6 Encoder ====<br />
* Specification: [[On2 VP6]]<br />
* Samples: http://samples.mplayerhq.hu/V-codecs/VP6/<br />
<br />
==== NUT Muxer ====<br />
* General improvements and enhancements<br />
<br />
==== DPX/Cineon Encoder AND Decoder ====<br />
* Specification: SMPTE 268M<br />
* http://en.wikipedia.org/wiki/DPX<br />
* Sources: [http://cinepaint.cvs.sourceforge.net/cinepaint/cinepaint-project/cinepaint/plug-ins/cineon/ CinePaint implementation] , [http://linuxmovies.org/ linuxmovies.org list of open source film tools]<br />
* Samples: ftp://ftp.graphicsmagick.org/pub/dpx/<br />
<br />
==== OpenEXR Encoder AND Decoder ====<br />
* Website and open source tools: http://openexr.com/<br />
* License: modified BSD<br />
* http://en.wikipedia.org/wiki/OpenEXR<br />
* Sources: [http://cinepaint.cvs.sourceforge.net/cinepaint/cinepaint-project/OpenEXR/ CinePaint implementation] , [http://linuxmovies.org/ linuxmovies.org list of open source film tools]<br />
<br />
==== HD Photo Encoder AND Decoder ====<br />
* Sources and specification: http://www.microsoft.com/whdc/xps/hdphotodpk.mspx<br />
* http://en.wikipedia.org/wiki/HD_Photo<br />
<br />
==== mp3PRO & aacPlus & MPEG Surround decoders ====<br />
* [http://www.codingtechnologies.com/products/mp3pro.htm mp3PRO] decoder. Note: mp3PRO decoding means MP3 + [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] demuxing/decoding. (Standard MP3 decoders can decode mp3PRO encoded files/streams but without [http://www.codingtechnologies.com/products/sbr.htm SBR] you do not get the full quality. By adding a [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder to FFmpeg and coupling it with the existing MP3 decoder you could playback mp3PRO at full quality? A [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder could be shared with a aacPlus decoder as aacPlus also uses [http://www.codingtechnologies.com/products/sbr.htm SBR]).<br />
* [http://www.codingtechnologies.com/products/aacPlus.htm aacPlus] (a.k.a. AAC+) decoder. Note: aacPlus v1 decoding means HE-AAC + [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] demuxing/decoding, and aacPlus v2 decoding means HE-AAC + [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] + [http://www.codingtechnologies.com/products/paraSter.htm PS (Parametric Stereo)] demuxing/decoding. (Standard HE-AAC decoders can decode aacPlus encoded files/streams but without [http://www.codingtechnologies.com/products/sbr.htm SBR] and [http://www.codingtechnologies.com/products/paraSter.htm PS] you do not get the full quality. By adding a [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder and a [http://www.codingtechnologies.com/products/paraSter.htm PS] decoder to FFmpeg and coupling it with an existing HE-AAC decoder you could playback aacPlus at full quality? A [http://www.codingtechnologies.com/products/sbr.htm SBR] decoder could be shared with a mp3PRO decoder as mp3PRO also uses [http://www.codingtechnologies.com/products/sbr.htm SBR]).<br />
* [http://www.mpegsurround.com MPEG Surround] decoder/parser (for all audio but especially MP3/mp3PRO and AAC/aacPlus as those are in use today). [http://www.codingtechnologies.com/products/mpgsrnd.htm MPEG Surround technology] share similar characteristics with [http://www.codingtechnologies.com/products/sbr.htm SBR (Spectral Band Replication)] and [http://www.codingtechnologies.com/products/paraSter.htm PS (Parametric Stereo)] demuxing/decoding, which [http://www.codingtechnologies.com/products/mp3pro.htm mp3PRO] and [http://www.codingtechnologies.com/products/aacPlus.htm aacPlus] decoders also use, so if SBR and PS decoders was added to FFmpeg then those could probebly share common code with a [http://www.mpegsurround.com MPEG Surround] decoder/parser. ([http://www.divx.com DivX Inc.] is one company that uses MPEG Surround technology to achieve 5.1 channel surround sound in smaller files).<br />
<br />
==== Native DirectShow support ====<br />
Decoder/encoder/demuxer/muxer and post-processing filter API for Windows by Microsoft<br />
* http://en.wikipedia.org/wiki/DirectShow<br />
http://en.wikipedia.org/wiki/Ffdshow<br />
* Specifications: http://msdn2.microsoft.com/en-us/library/ms783323.aspx<br />
**http://msdn2.microsoft.com/en-us/library/ms788119.aspx<br />
* Sources: http://sourceforge.net/projects/ffdshow-tryout/<br />
<br />
==== DirectX Video Acceleration (DXVA) 1.0 AND 2.0 for video decoding ====<br />
Note! For this native the above DirectShow support is first needed<br />
* http://en.wikipedia.org/wiki/DXVA<br />
* DXVA 1.0 (DirectX SDK) specifications: http://msdn2.microsoft.com/en-us/library/ms798379.aspx<br />
* DXVA 2.0 (Windows SDK) specifications: http://msdn2.microsoft.com/en-us/library/ms788119.aspx<br />
http://download.microsoft.com/download/5/b/9/5b97017b-e28a-4bae-ba48-174cf47d23cd/MED134_WH06.ppt<br />
<br />
==== Additional subtitle support ====<br />
* Create a common 'subtitles parser library' (and/or an API system for adding support for additional subtitle formats?) - a common sub-library to FFmpeg with all subtile decoders/demuxers/parsers gathered (similar to the libpostproc and libavutils). Call it "libsubs" (or "libsub", "libsubtitles" or whatever). Move FFmpeg's existing VobSub and DVBsub code there, so no matter if they are bitmap or text-based subs all existing and future subtile code is collected there. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional subtitles. <br />
**Maybe use MPlayer's recently added "'''libass'''" (SSA/ASS subtile reader) as a base for such a common library?<br />
* Support for advanced SSA/ASS rendering<br />
**Possible source are libass or the asa library <br />
* Support bold, italic, underline, RGB colors, size changes and font changes for a whole line or part of one line<br />
* [http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=284541 Line 23 signal (a.k.a. "Wide-screen signal")] detecting and use for DVD-Video (VobSub)<br />
* Support for the subtitles HTML tags<br />
* Capability of displaying subtitles with no video enabled (for example for audio-books)<br />
* Support for Karaoke subtitles (for kar and cdg, etc.)<br />
* Dual-subtitle-display (display two subtitles/languages at the same time, one at the bottom as normal plus one at the top of the screen)<br />
* Capability of moving the subtitles in the picture (freetype renderer)<br />
*Support more subtitle formats (text and bitmap-based):<br />
** [http://en.wikipedia.org/wiki/Closed_captioning Closed captioning (CC)] subtile support - (Closed captions for the deaf and hard of hearing, also known as "Line 21 captioning", uses VobSub bitmaps)<br />
***[http://www.xinehq.de xine] have a SPU decoder for subpictures and Closed Captions software decoding<br />
** DirectVobSub (VSFilter) - standard VobSubs (DVD-Video subtitles) embedded in AVI containers<br />
** [http://en.wikipedia.org/wiki/DivX#DivX_Subtitles_.28XSUB.29 DivX Subtitles (XSUB)] display/reader/decoder (Note: bitmap based subtitle, similar to VobSub)<br />
** [http://en.wikipedia.org/wiki/SubRip SubRip (.srt)] subtile support (Note: simple text-based based subtitle with timestamp)<br />
** Subviewer (.sub) subtile support (Note: simple text-based based subtitle with timestamp)<br />
** MicroDVD (.sub) subtile support (Note: simple text-based based subtitle with timestamp<br />
** [http://en.wikipedia.org/wiki/Sami_%28subtitle_tool%29 Sami (.smi)] subtile support (Note: simple text-based based subtitle with timestamp)<br />
** [http://en.wikipedia.org/wiki/SubStation_Alpha SubStation Alpha (.ssa+.ass)] subtile support (Note: advanced text-based based subtitle with timestamps and XY location on screen)<br />
** [http://en.wikipedia.org/wiki/Synchronized_Multimedia_Integration_Language RealText (.rt)] subtile support<br />
** PowerDivx (.psb) subtile support<br />
** [http://en.wikipedia.org/wiki/Universal_Subtitle_Format Universal Subtitle Format (.usf)] subtile support<br />
** Structured Subtitle Format (.ssf) subtile support<br />
<br />
==== libstream (a common 'stream client' library) ====<br />
* Create a common 'stream demuxer/parser library' for the client-side (and/or API for adding support for additional streaming formats?) - a LGPL'ed sub-library in FFmpeg with all stream demuxers/parsers gathered (similar to the libpostproc and libavutil). Call it "libstream" (or "stream" or whatever). Move FFmpeg's existing stream code there like HTTP and RTSP/RTP. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional streaming formats. All togther making it super easy for audio/video players using FFmpeg to add all-in-one streaming support to their player.<br />
**Maybe use either [http://www.mplayerhq.hu MPlayer]'s "''stream''" library structure, [http://www.live555.com LIVE555], or probebly the better [http://streaming.polito.it/client/library libnms] (from NeMeSi) as a base for such a common library?<br />
*Add support for additional streaming protocols (on the client side) and improve/enhance support for existing protocols:<br />
** HTTP (Hypertext Transfer Protocol) client<br />
** UDP (User Datagram Protocol) client<br />
** RTSP - Real-Time Streaming Protocol (RFC2326) client<br />
** RTP/RTCP - Real-Time Transport Protocol/RTP Control Protocol (RFC3550) client<br />
** RTP Profile for Audio and Video Conferences with Minimal Control (RFC3551) client<br />
** RealMedia RTSP/RDT (Real Time Streaming Protocol / Real Data Transport) client<br />
** SDP (Service Discovery Protocol) / SSDP (Simple Service Discovery Protocol) client<br />
** MMS (Microsoft Media Services) client<br />
<br />
== Qualification tasks ==<br />
<br />
Add a note if you choose to work on a Qualification task to avoid duplicate work.<br />
<br />
=== TIFF encoder ===<br />
* Specification: http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf <br />
* Samples: http://samples.mplayerhq.hu/mov/tiff/<br />
''Kamil Nowosad is working on this''<br />
<br />
''Bartlomiej Wolowiec is working on this''<br />
<br />
=== Vivo demuxer ===<br />
* Specification: look at the [[MPlayer]] [[vivo]] demuxer [http://svn.mplayerhq.hu/mplayer/trunk/libmpdemux/demux_viv.c?revision=22301&view=markup] <br />
* Samples: http://samples.mplayerhq.hu/vivo/ <br />
<br />
=== IFF/8SVX 8-bit audio demuxer ===<br />
* Specification: http://netghost.narod.ru/gff/vendspec/iff/iff.txt, http://sox.sourceforge.net/AudioFormats-11.html, and [http://xine.cvs.sourceforge.net/xine/xine-lib/src/demuxers/demux_iff.c?view=markup xine demuxer] <br />
* Samples: http://aminet.net/mods/smpl/<br />
<br />
''Tyler Williams is working on this task.''<br />
<br />
=== Port SGI image support to new API ===<br />
* Code: http://svn.mplayerhq.hu/ffmpeg/trunk/libavformat/sgi.c?revision=7867&view=markup&pathrev=7972<br />
FFmpeg changed image format APIs, but the SGI file format was never ported to the new API.<br />
**patch pending on ffmpeg-devel<br />
<br />
=== Optimize some code ===<br />
Do you think some code in FFmpeg could be made to run faster? We always love to get faster decoders or encoders.<br />
<br />
Note! This requires some ASM skills and using timer code to benchmark.<br />
<br />
Speedups via assembly optimizations (like SIMD for MMX, SSE/SSE2/SSE3 and Altivec) are needed in FFmpeg's:<br />
* [[H.264]] decoder<br />
* [[VC-1]] decoder<br />
* cat libavcodec/*.c | grep -i optimize for more files that need optimization.<br />
<br />
=== BFI Playback System ===<br />
Add FFmpeg playback capability for the [[BFI]] format. This entails writing a new file demuxer and simple video decoder.<br />
<br />
''DrV said he was working on this.''<br />
<br />
=== THP Playback System ===<br />
Add FFmpeg playback capability for the [[THP]] format. This entails writing a new file demuxer and leveraging existing JPEG and ADPCM decoders to handle the video and audio data inside.<br />
<br />
=== Bethsoft VID ===<br />
Add FFmpeg playback capability for the [[Bethsoft VID]] format (new demuxer, new video decoder).<br />
: <small>Looks like I might be able to get this...sorry for anyone who wanted it. --[[User:Gatoatigrado|Gatoatigrado]] 01:12, 24 March 2007 (EDT)</small><br />
<br />
=== Other Game Formats ===<br />
Several game formats are documented in this Wiki, but not yet implemented in FFmpeg. Investigate via the [[:Category:Game Formats]] page.<br />
<br />
=== Theora in Matroska ===<br />
The current mkv demuxer supports Vorbis but not Theora. Add support for Theora. This requires parsing the matroska extradata to extract the three header packets, and correctly passing these to the Theora decoder. You might want to read [http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2006-September/046120.html this thread] on the MPlayer list:<br />
<br />
''David Conrad is working on this''<br />
<br />
==See Also==<br />
*[[FFmpeg Wishlist]] for more tasks or ideas.<br />
*[[FFmpeg Summer Of Code|FFmpeg's Google SoC (Summer of Code) 2006 list of tasks]]<br />
*[http://bugzilla.mplayerhq.hu/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=FFmpeg&content= FFmpeg bugs] for ffmpeg bugs.</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_codec_HOWTO&diff=7347FFmpeg codec HOWTO2007-03-18T01:05:46Z<p>Gatoatigrado: /* libavcodec/allcodecs.c */</p>
<hr />
<div>This page is meant as an introduction to the internal codec API in [[FFmpeg]].<br />
It will also show how the codecs are connected with the demuxers. This is by<br />
no means a complete guide but enough to understand how to add a codec to FFmpeg.<br />
[[RealAudio cook|Cook]] is used as an example throughout.<br />
<br />
== libavcodec/avcodec.h ==<br />
The first thing to look at is the AVCodec struct.<br />
<br />
typedef struct AVCodec {<br />
const char *name;<br />
enum CodecType type;<br />
enum CodecID id;<br />
int priv_data_size;<br />
int (*init)(AVCodecContext *);<br />
int (*encode)(AVCodecContext *, uint8_t *buf, int buf_size, void *data);<br />
int (*close)(AVCodecContext *);<br />
int (*decode)(AVCodecContext *, void *outdata, int *outdata_size,<br />
uint8_t *buf, int buf_size);<br />
int capabilities;<br />
struct AVCodec *next;<br />
void (*flush)(AVCodecContext *);<br />
const AVRational *supported_framerates; ///array of supported framerates, or NULL if any, array is terminated by {0,0}<br />
const enum PixelFormat *pix_fmts; ///array of supported pixel formats, or NULL if unknown, array is terminanted by -1<br />
} AVCodec;<br />
<br />
Here we can see that we have some elements to name the codec, what type it is (audio/video), the supported pixel formats and some function<br />
pointers for init/encode/decode and close. Now lets see how it is used.<br />
<br />
== libavcodec/cook.c ==<br />
If we look in this file at the bottom we can see this code:<br />
<br />
AVCodec cook_decoder =<br />
{<br />
.name = "cook",<br />
.type = CODEC_TYPE_AUDIO,<br />
.id = CODEC_ID_COOK,<br />
.priv_data_size = sizeof(COOKContext),<br />
.init = cook_decode_init,<br />
.close = cook_decode_close,<br />
.decode = cook_decode_frame,<br />
};<br />
<br />
First we get an AVCodec struct named cook_decoder. And then we set the variables of cook_decoder. Note that we only set the variables that are needed. Currently there is no encoder so we don't set any. If we now look at the id variable we can see that CODEC_ID_COOK isn't defined in libavcodec/cook.c. It is declared in avcodec.h.<br />
<br />
== libavcodec/avcodec.h ==<br />
<br />
Here we will find the CodecID enumeration.<br />
<br />
enum CodecID {<br />
...<br />
CODEC_ID_GSM,<br />
CODEC_ID_QDM2,<br />
CODEC_ID_COOK,<br />
CODEC_ID_TRUESPEECH,<br />
CODEC_ID_TTA,<br />
...<br />
};<br />
<br />
CODEC_ID_COOK is there in the list. This is the list of all supported codecs in FFmpeg, the list is fixed and used internally to id every codec. Changing the order would break binary compatibility.<br />
<br />
This is all enough to declare a codec. Now we must register them for internal use also. This is done at runtime.<br />
<br />
== libavcodec/allcodecs.c ==<br />
In this file we have the avcodec_register_all() function, it has entries like this for all codecs.<br />
<br />
...<br />
REGISTER_DECODER(COOK, cook);<br />
...<br />
<br />
This macro expands to a register_avcodec() call which registers a codec for internal use.<br />
Note that register_avcodec() will only be called when CONFIG_COOK_DECODER is defined.<br />
This allows to not compile the decoder code for a specific codec.<br />
But where is it defined? This is extracted by configure with this command line:<br />
<br />
sed -n 's/^[^#]*DEC.*, *\(.*\)).*/\1_decoder/p' libavcodec/allcodecs.c<br />
<br />
So adding a REGISTER_DECODER(NEW, new) entry in allcodecs.c and reconfigure is enough to add the needed define.<br />
One more thing to note is that cook.c isn't included in allcodecs.c so the symbol cook_decoder can't be<br />
found. Thus it has to be declared somewhere, that happens in <code>avcodec.h</code> and it is declared as an external symbol.<br />
extern AVCodec cook_decoder;<br />
<br />
Now we have everything to hookup a codec now we will see how the codec. For this we look into libavformat.<br />
<br />
== libavformat/rm.c ==<br />
If we think of an imaginary rm file that ffmpeg is about to process, the first thing that happens is that it is identified<br />
as a rm file. It is passed on to the rm demuxer (rm.c). The rm demuxer looks through the file and finds out that it is a<br />
cook file.<br />
<br />
...<br />
} else if (!strcmp(buf, "cook")) {<br />
st->codec->codec_id = CODEC_ID_COOK;<br />
...<br />
<br />
Now ffmpeg knows what codec to init and where to send the payload from the container. So back to cook.c and the initialization process.<br />
<br />
== libavcodec/cook.c Init ==<br />
After ffmpeg knows what codec to use, it calls the declared initialization function pointer declared in the codecs AVCodec struct. In<br />
cook.c it is called cook_decode_init. Here we setup as much as we can before we start decoding. The following things should be handled in the init, vlc table initialization, table generation, memory allocation and extradata parsing.<br />
<br />
== libavcodec/cook.c Close ==<br />
The cook_decode_close function is the codec clean-up call. All memory, vlc tables, etc. should be freed here.<br />
<br />
== libavcodec/cook.c Decode ==<br />
In cook.c the name of the decode call is cook_decode_frame.<br />
<br />
static int cook_decode_frame(AVCodecContext *avctx,<br />
void *data, int *data_size,<br />
uint8_t *buf, int buf_size) {<br />
...<br />
<br />
The function has 5 arguments:<br />
* avctx is a pointer to a AVCodecContext<br />
* data is the pointer to the outbuffer<br />
* data_size is a variable that should be set to the outbuffer size in bytes<br />
* buf is the pointer to the inbuffer<br />
* buf_size is the size of the inbuffer<br />
<br />
The decode function shall return the amount of bytes consumed from the inbuffer.<br />
<br />
<br />
That's how it works without too much detail.<br />
<br />
<br />
== The Glue codec template ==<br />
The imaginary Glue audio codec will serve as a base to exhibit bitstream reading, vlc decoding and other things.<br />
The code is purely fictional and is sometimes written purely for the sake of example. No attempt is made to prevent invalid<br />
data manipulation.<br />
<br />
The Glue codec follows.<br />
<br />
/* First we include some default includes */<br />
#include <math.h><br />
#include <stddef.h><br />
#include <stdio.h><br />
<br />
/* The following includes have the bitstream reader, various dsp functions and the various defaults */<br />
#define ALT_BITSTREAM_READER<br />
#include "avcodec.h"<br />
#include "bitstream.h"<br />
#include "dsputil.h"<br />
<br />
/* This includes the tables needed for the Glue codec template */<br />
#include "gluedata.h"<br />
<br />
<br />
/* Here we declare the struct used for the codec private data */<br />
typedef struct {<br />
GetBitContext gb;<br />
FFTContext fft_ctx;<br />
VLC vlc_table;<br />
MDCTContext mdct_ctx;<br />
float* sample_buffer;<br />
} GLUEContext;<br />
<br />
<br />
/* The init function */<br />
static int glue_decode_init(AVCodecContext *avctx)<br />
{<br />
GLUEContext *q = avctx->priv_data;<br />
<br />
/* This imaginary codec uses one fft, one mdct and one vlc table. */<br />
ff_mdct_init(&q->mdct_ctx, 10, 1); // 2^10 == size of mdct, 1 == inverse mdct<br />
ff_fft_init(&q->fft_ctx, 9, 1); // 2^9 == size of fft, 0 == inverse fft<br />
init_vlc (&q->vlc_table, 9, 24,<br />
vlctable_huffbits, 1, 1,<br />
vlctable_huffcodes, 2, 2, 0); // look in bitstream.h for the meaning of the arguments<br />
<br />
/* We also need to allocate a sample buffer */<br />
q->sample_buffer = av_mallocz(sizeof(float)*1024); // here we used av_mallocz instead of av_malloc<br />
// av_mallocz memsets the whole buffer to 0<br />
<br />
/* Check if the allocation was successful */<br />
if(q->sample_buffer == NULL)<br />
return -1;<br />
<br />
/* return 0 for a successful init, -1 for failure */<br />
return 0;<br />
}<br />
<br />
<br />
/* This is the main decode function */<br />
static int glue_decode_frame(AVCodecContext *avctx,<br />
void *data, int *data_size,<br />
uint8_t *buf, int buf_size)<br />
{<br />
GLUEContext *q = avctx->priv_data;<br />
int16_t *outbuffer = data;<br />
<br />
/* We know what the arguments for this function are from above<br />
now we just have to decode this imaginary codec, the made up<br />
bitstream format is as follows:<br />
12 bits representing the amount of samples<br />
1 bit fft or mdct coded coeffs, 0 for fft/1 for mdct<br />
read 13 bits representing the amount of vlc coded fft data coeffs<br />
read 10 bits representing the amount of vlc coded mdct data coeffs<br />
(...bits representing the coeffs...)<br />
5 bits of dummy data that should be ignored<br />
32 bits the hex value 0x12345678, used for integrity check<br />
*/<br />
<br />
/* Declare the needed variables */<br />
int samples, coeffs, i, fft;<br />
float mdct_tmp[1024];<br />
<br />
/* Now we init the bitstream reader, we start at the beginning of the inbuffer */<br />
init_get_bits(&q->gb, buf, buf_size*8); //the buf_size is in bytes but we need bits<br />
<br />
/* Now we take 12 bits to get the amount of samples the current frame has */<br />
samples = get_bits(&q->gb, 12);<br />
<br />
/* Now we check if we have fft or mdct coeffs */<br />
fft = get_bits1(&q->gb);<br />
if (fft) {<br />
//fft coeffs, get how many<br />
coeffs = get_bits(&q->gb, 13);<br />
} else {<br />
//mdct coeffs, get how many<br />
coeffs = get_bits(&q->gb, 10);<br />
}<br />
<br />
/* Now decode the vlc coded coeffs to the sample_buffer */<br />
for (i=0 ; i<coeffs ; i++)<br />
q->sample_buffer[i] = get_vlc2(&q->gb, q->vlc_table.table, vlc_table.bits, 3); //read about the arguments in bitstream.h<br />
<br />
/* Now we need to transform the coeffs to samples */<br />
if (fft) {<br />
//The fft is done inplace<br />
ff_fft_permute(&q->fft_ctx, (FFTComplex *) q->sample_buffer);<br />
ff_fft_calc(&q->fft_ctx, (FFTComplex *) q->sample_buffer);<br />
} else {<br />
//And we pretend that the mdct is also inplace<br />
ff_imdct_calc(&q->mdct_ctx, q->sample_buffer, q->sample_buffer, mdct_tmp);<br />
}<br />
<br />
/* To make it easy the stream can only be 16 bits mono, so let's convert it to that */<br />
for (i=0 ; i<samples ; i++)<br />
outbuffer[i] = (int16_t)q->sample_buffer[i];<br />
<br />
/* Report how many samples we got */<br />
*data_size = samples;<br />
<br />
/* Skip the dummy data bits */<br />
skip_bits(&q->gb, 5);<br />
<br />
/* Check if the buffer was consumed ok */<br />
if (get_bits(&q->gb,32) != 0x12345678) {<br />
av_log(avctx,AV_LOG_ERROR,"Stream error, integrity check failed!\n");<br />
return -1;<br />
}<br />
<br />
/* Return the amount of bytes consumed if everything was ok */<br />
return *data_size*sizeof(int16_t);<br />
}<br />
<br />
<br />
/* the uninit function, here we just do the inverse of the init */ <br />
static int glue_decode_close(AVCodecContext *avctx)<br />
{<br />
GLUEContext *q = avctx->priv_data;<br />
<br />
/* Free allocated memory buffer */<br />
av_free(q->sample_buffer);<br />
<br />
/* Free the fft transform */<br />
ff_fft_end(&q->fft_ctx);<br />
<br />
/* Free the mdct transform */<br />
ff_mdct_end(&q->mdct_ctx);<br />
<br />
/* Free the vlc table */<br />
free_vlc(&q->vlc_table);<br />
<br />
/* Return 0 if everything is ok, -1 if not */<br />
return 0;<br />
}<br />
<br />
<br />
AVCodec glue_decoder =<br />
{<br />
.name = "glue",<br />
.type = CODEC_TYPE_AUDIO,<br />
.id = CODEC_ID_GLUE,<br />
.priv_data_size = sizeof(GLUEContext),<br />
.init = glue_decode_init,<br />
.close = glue_decode_close,<br />
.decode = glue_decode_frame,<br />
};</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=Talk:FFmpeg_codec_HOWTO&diff=7346Talk:FFmpeg codec HOWTO2007-03-18T01:04:22Z<p>Gatoatigrado: </p>
<hr />
<div>I have a few questions. I can modify the wiki page when I find these out (maybe through email?)<br />
<br />
1.) need to modify the makefile?<br />
<br />
entries such as<br />
OBJS-$(CONFIG_AASC_DECODER) += aasc.o<br />
<br />
The command line utility -- whatever that did -- didn't do this.<br />
<br />
Thanks. --[[User:Gatoatigrado|Gatoatigrado]] 20:57, 17 March 2007 (EDT)</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=Talk:FFmpeg_codec_HOWTO&diff=7345Talk:FFmpeg codec HOWTO2007-03-18T00:57:10Z<p>Gatoatigrado: </p>
<hr />
<div>I have a few questions. I can modify the wiki page when I find these out (maybe through email?)<br />
<br />
1.) need to modify the makefile?<br />
<br />
entries such as<br />
OBJS-$(CONFIG_AASC_DECODER) += aasc.o<br />
<br />
The command line utility -- whatever that did -- didn't do this.<br />
<br />
2.) What about these? gcc gives compilation errors; it needs to know to look for the symbol in the codec's c file<br />
<br />
avcodec.h<br />
extern AVCodec ac3_encoder;<br />
extern AVCodec amr_nb_encoder;<br />
extern AVCodec amr_wb_encoder;<br />
<br />
Thanks. --[[User:Gatoatigrado|Gatoatigrado]] 20:57, 17 March 2007 (EDT)</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_codec_HOWTO&diff=7344FFmpeg codec HOWTO2007-03-18T00:14:01Z<p>Gatoatigrado: </p>
<hr />
<div>This page is meant as an introduction to the internal codec API in [[FFmpeg]].<br />
It will also show how the codecs are connected with the demuxers. This is by<br />
no means a complete guide but enough to understand how to add a codec to FFmpeg.<br />
[[RealAudio cook|Cook]] is used as an example throughout.<br />
<br />
== libavcodec/avcodec.h ==<br />
The first thing to look at is the AVCodec struct.<br />
<br />
typedef struct AVCodec {<br />
const char *name;<br />
enum CodecType type;<br />
enum CodecID id;<br />
int priv_data_size;<br />
int (*init)(AVCodecContext *);<br />
int (*encode)(AVCodecContext *, uint8_t *buf, int buf_size, void *data);<br />
int (*close)(AVCodecContext *);<br />
int (*decode)(AVCodecContext *, void *outdata, int *outdata_size,<br />
uint8_t *buf, int buf_size);<br />
int capabilities;<br />
struct AVCodec *next;<br />
void (*flush)(AVCodecContext *);<br />
const AVRational *supported_framerates; ///array of supported framerates, or NULL if any, array is terminated by {0,0}<br />
const enum PixelFormat *pix_fmts; ///array of supported pixel formats, or NULL if unknown, array is terminanted by -1<br />
} AVCodec;<br />
<br />
Here we can see that we have some elements to name the codec, what type it is (audio/video), the supported pixel formats and some function<br />
pointers for init/encode/decode and close. Now lets see how it is used.<br />
<br />
== libavcodec/cook.c ==<br />
If we look in this file at the bottom we can see this code:<br />
<br />
AVCodec cook_decoder =<br />
{<br />
.name = "cook",<br />
.type = CODEC_TYPE_AUDIO,<br />
.id = CODEC_ID_COOK,<br />
.priv_data_size = sizeof(COOKContext),<br />
.init = cook_decode_init,<br />
.close = cook_decode_close,<br />
.decode = cook_decode_frame,<br />
};<br />
<br />
First we get an AVCodec struct named cook_decoder. And then we set the variables of cook_decoder. Note that we only set the variables that are needed. Currently there is no encoder so we don't set any. If we now look at the id variable we can see that CODEC_ID_COOK isn't defined in libavcodec/cook.c. It is declared in avcodec.h.<br />
<br />
== libavcodec/avcodec.h ==<br />
<br />
Here we will find the CodecID enumeration.<br />
<br />
enum CodecID {<br />
...<br />
CODEC_ID_GSM,<br />
CODEC_ID_QDM2,<br />
CODEC_ID_COOK,<br />
CODEC_ID_TRUESPEECH,<br />
CODEC_ID_TTA,<br />
...<br />
};<br />
<br />
CODEC_ID_COOK is there in the list. This is the list of all supported codecs in FFmpeg, the list is fixed and used internally to id every codec. Changing the order would break binary compatibility.<br />
<br />
This is all enough to declare a codec. Now we must register them for internal use also. This is done at runtime.<br />
<br />
== libavcodec/allcodecs.c ==<br />
In this file we have the avcodec_register_all() function, it has entries like this for all codecs.<br />
<br />
...<br />
REGISTER_DECODER(COOK, cook);<br />
...<br />
<br />
This macro expands to a register_avcodec() call which registers a codec for internal use.<br />
Note that register_avcodec() will only be called when CONFIG_COOK_DECODER is defined.<br />
This allows to not compile the decoder code for a specific codec.<br />
But where is it defined? This is extracted by configure with this command line:<br />
<br />
sed -n 's/^[^#]*DEC.*, *\(.*\)).*/\1_decoder/p' libavcodec/allcodecs.c<br />
<br />
So adding a REGISTER_DECODER(NEW, new) entry in allcodecs.c and reconfigure is enough to add the needed define.<br />
One more thing to note is that cook.c isn't included in allcodecs.c so the symbol cook_decoder can't be<br />
found. Thus it has to be declared somewhere, that happens in avcodec.h and it is declared as an external symbol.<br />
<br />
Now we have everything to hookup a codec now we will see how the codec. For this we look into libavformat.<br />
<br />
== libavformat/rm.c ==<br />
If we think of an imaginary rm file that ffmpeg is about to process, the first thing that happens is that it is identified<br />
as a rm file. It is passed on to the rm demuxer (rm.c). The rm demuxer looks through the file and finds out that it is a<br />
cook file.<br />
<br />
...<br />
} else if (!strcmp(buf, "cook")) {<br />
st->codec->codec_id = CODEC_ID_COOK;<br />
...<br />
<br />
Now ffmpeg knows what codec to init and where to send the payload from the container. So back to cook.c and the initialization process.<br />
<br />
== libavcodec/cook.c Init ==<br />
After ffmpeg knows what codec to use, it calls the declared initialization function pointer declared in the codecs AVCodec struct. In<br />
cook.c it is called cook_decode_init. Here we setup as much as we can before we start decoding. The following things should be handled in the init, vlc table initialization, table generation, memory allocation and extradata parsing.<br />
<br />
== libavcodec/cook.c Close ==<br />
The cook_decode_close function is the codec clean-up call. All memory, vlc tables, etc. should be freed here.<br />
<br />
== libavcodec/cook.c Decode ==<br />
In cook.c the name of the decode call is cook_decode_frame.<br />
<br />
static int cook_decode_frame(AVCodecContext *avctx,<br />
void *data, int *data_size,<br />
uint8_t *buf, int buf_size) {<br />
...<br />
<br />
The function has 5 arguments:<br />
* avctx is a pointer to a AVCodecContext<br />
* data is the pointer to the outbuffer<br />
* data_size is a variable that should be set to the outbuffer size in bytes<br />
* buf is the pointer to the inbuffer<br />
* buf_size is the size of the inbuffer<br />
<br />
The decode function shall return the amount of bytes consumed from the inbuffer.<br />
<br />
<br />
That's how it works without too much detail.<br />
<br />
<br />
== The Glue codec template ==<br />
The imaginary Glue audio codec will serve as a base to exhibit bitstream reading, vlc decoding and other things.<br />
The code is purely fictional and is sometimes written purely for the sake of example. No attempt is made to prevent invalid<br />
data manipulation.<br />
<br />
The Glue codec follows.<br />
<br />
/* First we include some default includes */<br />
#include <math.h><br />
#include <stddef.h><br />
#include <stdio.h><br />
<br />
/* The following includes have the bitstream reader, various dsp functions and the various defaults */<br />
#define ALT_BITSTREAM_READER<br />
#include "avcodec.h"<br />
#include "bitstream.h"<br />
#include "dsputil.h"<br />
<br />
/* This includes the tables needed for the Glue codec template */<br />
#include "gluedata.h"<br />
<br />
<br />
/* Here we declare the struct used for the codec private data */<br />
typedef struct {<br />
GetBitContext gb;<br />
FFTContext fft_ctx;<br />
VLC vlc_table;<br />
MDCTContext mdct_ctx;<br />
float* sample_buffer;<br />
} GLUEContext;<br />
<br />
<br />
/* The init function */<br />
static int glue_decode_init(AVCodecContext *avctx)<br />
{<br />
GLUEContext *q = avctx->priv_data;<br />
<br />
/* This imaginary codec uses one fft, one mdct and one vlc table. */<br />
ff_mdct_init(&q->mdct_ctx, 10, 1); // 2^10 == size of mdct, 1 == inverse mdct<br />
ff_fft_init(&q->fft_ctx, 9, 1); // 2^9 == size of fft, 0 == inverse fft<br />
init_vlc (&q->vlc_table, 9, 24,<br />
vlctable_huffbits, 1, 1,<br />
vlctable_huffcodes, 2, 2, 0); // look in bitstream.h for the meaning of the arguments<br />
<br />
/* We also need to allocate a sample buffer */<br />
q->sample_buffer = av_mallocz(sizeof(float)*1024); // here we used av_mallocz instead of av_malloc<br />
// av_mallocz memsets the whole buffer to 0<br />
<br />
/* Check if the allocation was successful */<br />
if(q->sample_buffer == NULL)<br />
return -1;<br />
<br />
/* return 0 for a successful init, -1 for failure */<br />
return 0;<br />
}<br />
<br />
<br />
/* This is the main decode function */<br />
static int glue_decode_frame(AVCodecContext *avctx,<br />
void *data, int *data_size,<br />
uint8_t *buf, int buf_size)<br />
{<br />
GLUEContext *q = avctx->priv_data;<br />
int16_t *outbuffer = data;<br />
<br />
/* We know what the arguments for this function are from above<br />
now we just have to decode this imaginary codec, the made up<br />
bitstream format is as follows:<br />
12 bits representing the amount of samples<br />
1 bit fft or mdct coded coeffs, 0 for fft/1 for mdct<br />
read 13 bits representing the amount of vlc coded fft data coeffs<br />
read 10 bits representing the amount of vlc coded mdct data coeffs<br />
(...bits representing the coeffs...)<br />
5 bits of dummy data that should be ignored<br />
32 bits the hex value 0x12345678, used for integrity check<br />
*/<br />
<br />
/* Declare the needed variables */<br />
int samples, coeffs, i, fft;<br />
float mdct_tmp[1024];<br />
<br />
/* Now we init the bitstream reader, we start at the beginning of the inbuffer */<br />
init_get_bits(&q->gb, buf, buf_size*8); //the buf_size is in bytes but we need bits<br />
<br />
/* Now we take 12 bits to get the amount of samples the current frame has */<br />
samples = get_bits(&q->gb, 12);<br />
<br />
/* Now we check if we have fft or mdct coeffs */<br />
fft = get_bits1(&q->gb);<br />
if (fft) {<br />
//fft coeffs, get how many<br />
coeffs = get_bits(&q->gb, 13);<br />
} else {<br />
//mdct coeffs, get how many<br />
coeffs = get_bits(&q->gb, 10);<br />
}<br />
<br />
/* Now decode the vlc coded coeffs to the sample_buffer */<br />
for (i=0 ; i<coeffs ; i++)<br />
q->sample_buffer[i] = get_vlc2(&q->gb, q->vlc_table.table, vlc_table.bits, 3); //read about the arguments in bitstream.h<br />
<br />
/* Now we need to transform the coeffs to samples */<br />
if (fft) {<br />
//The fft is done inplace<br />
ff_fft_permute(&q->fft_ctx, (FFTComplex *) q->sample_buffer);<br />
ff_fft_calc(&q->fft_ctx, (FFTComplex *) q->sample_buffer);<br />
} else {<br />
//And we pretend that the mdct is also inplace<br />
ff_imdct_calc(&q->mdct_ctx, q->sample_buffer, q->sample_buffer, mdct_tmp);<br />
}<br />
<br />
/* To make it easy the stream can only be 16 bits mono, so let's convert it to that */<br />
for (i=0 ; i<samples ; i++)<br />
outbuffer[i] = (int16_t)q->sample_buffer[i];<br />
<br />
/* Report how many samples we got */<br />
*data_size = samples;<br />
<br />
/* Skip the dummy data bits */<br />
skip_bits(&q->gb, 5);<br />
<br />
/* Check if the buffer was consumed ok */<br />
if (get_bits(&q->gb,32) != 0x12345678) {<br />
av_log(avctx,AV_LOG_ERROR,"Stream error, integrity check failed!\n");<br />
return -1;<br />
}<br />
<br />
/* Return the amount of bytes consumed if everything was ok */<br />
return *data_size*sizeof(int16_t);<br />
}<br />
<br />
<br />
/* the uninit function, here we just do the inverse of the init */ <br />
static int glue_decode_close(AVCodecContext *avctx)<br />
{<br />
GLUEContext *q = avctx->priv_data;<br />
<br />
/* Free allocated memory buffer */<br />
av_free(q->sample_buffer);<br />
<br />
/* Free the fft transform */<br />
ff_fft_end(&q->fft_ctx);<br />
<br />
/* Free the mdct transform */<br />
ff_mdct_end(&q->mdct_ctx);<br />
<br />
/* Free the vlc table */<br />
free_vlc(&q->vlc_table);<br />
<br />
/* Return 0 if everything is ok, -1 if not */<br />
return 0;<br />
}<br />
<br />
<br />
AVCodec glue_decoder =<br />
{<br />
.name = "glue",<br />
.type = CODEC_TYPE_AUDIO,<br />
.id = CODEC_ID_GLUE,<br />
.priv_data_size = sizeof(GLUEContext),<br />
.init = glue_decode_init,<br />
.close = glue_decode_close,<br />
.decode = glue_decode_frame,<br />
};</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_Of_Code_2007&diff=7341FFmpeg Summer Of Code 20072007-03-17T22:13:17Z<p>Gatoatigrado: /* Game Formats */</p>
<hr />
<div>Google is sponsoring their third annual [http://code.google.com/soc/ Summer of Code] for the summer of 2007. This entails sponsoring students to work on assorted open source projects as well as sponsoring mentors in those same projects. Everyone wins.<br />
<br />
[[FFmpeg]] was a Summer of Code participant in the [http://code.google.com/soc/2006/ffmpeg/about.html summer of 2006] (here is the [[FFmpeg Summer Of Code|corresponding Wiki page]]).<br />
<br />
[[User:Multimedia Mike|Mike Melanson]] (mike -at- multimedia.cx) is the administrator and main point of contact for matters relating to the FFmpeg Summer of Code.<br />
<br />
== How to apply ==<br />
<br />
Before you can apply make sure you are qualified enough to apply. Last year 50% of the applicants weren't qualified for the task they applied for.<br />
<br />
* You have to know how to program in C fairly well.<br />
* We would like you to submit a patch that fixes a bug or adds a feature to FFmpeg. By doing that we will know that you are qualified for the task or not. On this page there is a list of [[Summer Of Code 2007#Qualification_tasks|Qualification Tasks]] that can be done. But you are free to submit anything you feel might be of value to FFmpeg. The qualification task can be done after you have filed you application.<br />
* Submit a good application through the formal Google Summer of Code process during the application timeframe (March 14-24, 2007).<br />
* You have to have >35 hours per week to put into the project.<br />
* You can't have another job at the same time as the SoC project.<br />
<br />
== Current Status ==<br />
* March 5-12, 2007: Application period for mentoring organizations.<br />
* March 6, 2007: Mike Melanson submitted FFmpeg mentoring application.<br />
* March 14, 2007: Google are now accepting student Applications.<br />
* March 15, 2007: FFmpeg got accepted as mentoring organization; now accepting student applications until March 24, 2007.<br />
<br />
== Project Proposals ==<br />
<br />
=== Overview ===<br />
Qualifications for a good Summer of Code proposal:<br />
* discrete, well-defined, modular<br />
* comprised of a series of measurable sub-goals<br />
* based on open specs that are available free of charge<br />
* based on complete specs<br />
An example of a good proposal is the implementation of a decoder or demuxer for an as yet unsupported multimedia format, or an encoder or muxer for a format that can already be demuxed/decoded by FFmpeg.<br />
<br />
An example of a less desirable proposal is one that's not as measurable, such as refactoring APIs. Bad proposals tend to be ones that would require touching a lot of core code.<br />
<br />
To re-iterate:<br />
* Localized/isolated code projects = ''good''<br />
* Global code refactoring = ''bad''<br />
<br />
=== Note ===<br />
THIS LIST IS NOT THE PROPOSALS WE ARE SUBMITTING!<br />
<br />
Most of this list is just some ideas we are kicking around.<br />
<br />
=== Projects with Mentors (official projects) ===<br />
<br />
==== QCELP Decoder ====<br />
<br />
* Specification: QCELP decoder [http://www.3gpp2.org/Public_html/specs/alltsgscfm.cfm spec] is c.s0020 and source is c.r0020<br />
* Sample files: http://samples.mplayerhq.hu/A-codecs/qclp/<br />
<br />
''Mentor: Benjamin Larsson''<br />
<br />
==== Matroska Muxer ====<br />
<br />
* Specification: http://www.matroska.org/technical/specs/index.html<br />
* Sample files: http://samples.mplayerhq.hu/Matroska/<br />
<br />
''Mentor: Aurelien Jacobs; Backup mentors: Steve Lhomme, Ronald S. Bultje''<br />
<br />
==== MPEG TS/DVB Muxer ====<br />
* Specification: ISO 13818-1<br />
<br />
''Mentor: Baptiste Coudurier''<br />
<br />
==== MXF Muxer ====<br />
* Specification: SMPTE 377M<br />
<br />
''Mentor: Baptiste Coudurier''<br />
<br />
==== RV40 Decoder ====<br />
* [[RealVideo 4]] is steadily being reverse engineered and should be a reasonable candidate for re-implementation by the summer.<br />
<br />
''Mentor: Mike Melanson''<br />
<br />
==== PAFF decoding for H.264/AVC ====<br />
* Specification: [http://www.itu.int/rec/T-REC-H.264-200503-I/en ITU-T]<br />
* Sample files: http://samples.mplayerhq.hu/V-codecs/h264/PAFF/<br />
<br />
''Mentor: Loren Merritt''<br />
<br />
==== Dirac Encoder AND Decoder ====<br />
* Website: http://dirac.sf.net<br />
* Specification: http://dirac.sourceforge.net/specification.html<br />
<br />
''Mentor: Luca Barbato''<br />
<br />
==== E-AC3 Decoder ====<br />
* Specification: http://www.atsc.org/standards/a_52b.pdf<br />
* Samples: http://samples.mplayerhq.hu/evob/MAININTRO.EVO<br />
<br />
''Mentor: Justin Ruggles''<br />
<br />
=== Ideas for more projects to be determined ===<br />
<br />
==== JPEG2000 ====<br />
* Specifications: [http://www.itu.int/rec/T-REC-T/e As ITU-T recommendations], [http://isotc.iso.org/livelink/livelink/fetch/2000/2489/Ittf_Home/PubliclyAvailableStandards.htm ISO publicly available standards]<br />
* Sources: [http://www.ece.uvic.ca/~mdadams/jasper/ JasPer], [http://www.openjpeg.org/ OpenJpeg], [http://jj2000.epfl.ch/ JJ2000 (in Java)]<br />
* Samples: http://samples.mplayerhq.hu/jpeg2000/<br />
<br />
==== Monkey's Audio Decoder ====<br />
* Sources: [http://www.monkeysaudio.com/files/MAC_SDK_399.zip original sources], [http://sourceforge.net/projects/mac-port/ original sources port for non-win32 platforms], [http://jmac.sourceforge.net/ LGPLed Java implementation]<br />
<br />
==== Finish LC AAC decoder and implement HE-AAC(+) ====<br />
* Code: http://svn.mplayerhq.hu/aac/<br />
<br />
<br />
==== GSM Decoder ====<br />
* Specification + sample implementation: http://kbs.cs.tu-berlin.de/~jutta/toast.html<br />
* Samples: http://samples.mplayerhq.hu/A-codecs/GSM/<br />
<br />
==== i263 Decoder ====<br />
* Specification: [[I263 | I263 Format Specification at MultimediaWiki]]<br />
* Sources: [http://multimedia.cx/I263Src.zip GPLed I263 decoder]<br />
* Sample files: http://samples.mplayerhq.hu/V-codecs/I263/<br />
<br />
==== VP6 Encoder ====<br />
* Specification: [[On2 VP6]]<br />
* Samples: http://samples.mplayerhq.hu/V-codecs/VP6/<br />
<br />
==== NUT Muxer ====<br />
<br />
==== DPX/Cineon Encoder AND Decoder ====<br />
* Specification: SMPTE 268M<br />
* http://en.wikipedia.org/wiki/DPX<br />
* Sources: [http://cinepaint.cvs.sourceforge.net/cinepaint/cinepaint-project/cinepaint/plug-ins/cineon/ CinePaint implementation] , [http://linuxmovies.org/ linuxmovies.org list of open source film tools]<br />
* Samples: ftp://ftp.graphicsmagick.org/pub/dpx/<br />
<br />
==== OpenEXR Encoder AND Decoder ====<br />
* Website and open source tools: http://openexr.com/<br />
* License: modified BSD<br />
* http://en.wikipedia.org/wiki/OpenEXR<br />
* Sources: [http://cinepaint.cvs.sourceforge.net/cinepaint/cinepaint-project/OpenEXR/ CinePaint implementation] , [http://linuxmovies.org/ linuxmovies.org list of open source film tools]<br />
<br />
==== HD Photo Encoder AND Decoder ====<br />
* Sources and specification: http://www.microsoft.com/whdc/xps/hdphotodpk.mspx<br />
* http://en.wikipedia.org/wiki/HD_Photo<br />
<br />
== Qualification tasks ==<br />
<br />
Add a note if you choose to work on a Qualification task to avoid duplicate work.<br />
<br />
=== TIFF encoder ===<br />
* Specification: http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf <br />
* Samples: http://samples.mplayerhq.hu/mov/tiff/ <br />
<br />
=== Vivo demuxer ===<br />
* Specification: look at the [[MPlayer]] [[vivo]] demuxer [http://svn.mplayerhq.hu/mplayer/trunk/libmpdemux/demux_viv.c?revision=22301&view=markup] <br />
* Samples: http://samples.mplayerhq.hu/vivo/ <br />
<br />
=== IFF/8SVX 8-bit audio demuxer ===<br />
* Specification: http://netghost.narod.ru/gff/vendspec/iff/iff.txt, http://sox.sourceforge.net/AudioFormats-11.html, and [http://xine.cvs.sourceforge.net/xine/xine-lib/src/demuxers/demux_iff.c?view=markup xine demuxer] <br />
* Samples: http://aminet.net/mods/smpl/<br />
<br />
=== Port SGI image support to new API ===<br />
* Code: http://svn.mplayerhq.hu/ffmpeg/trunk/libavformat/sgi.c?revision=7867&view=markup&pathrev=7972<br />
FFmpeg changed image format APIs, but the SGI file format was never ported to the new API.<br />
<br />
=== Optimize some code ===<br />
Do you think some code in ffmpeg could run faster? We would love to get some faster decoders or encoders.<br />
<br />
This requires some ASM skills and using timer code to benchmark.<br />
<br />
Speedups needed in:<br />
* h264 decoder<br />
* vc-1 decoder<br />
<br />
<br />
=== BFI Playback System ===<br />
Add FFmpeg playback capability for the [[BFI]] format. This entails writing a new file demuxer and simple video decoder.<br />
<br />
''DrV said he was working on this.''<br />
<br />
=== THP Playback System ===<br />
Add FFmpeg playback capability for the [[THP]] format. This entails writing a new file demuxer and leveraging existing JPEG and ADPCM decoders to handle the video and audio data inside.<br />
<br />
=== Game Formats ===<br />
Several game formats are documented in this Wiki, but not yet implemented in FFmpeg. Some of them are [[Bethsoft VID]], ...<br />
: <small>I will take a shot at VID, but anyone else should feel free to try --[[User:Gatoatigrado|Gatoatigrado]] 18:13, 17 March 2007 (EDT)</small><br />
<br />
=== Theora in Matroska ===<br />
The current mkv demuxer supports Vorbis but not Theora. Add support for Theora. This requires parsing the matroska extradata to extract the three header packets, and correctly passing these to the Theora decoder. You might want to read [http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2006-September/046120.html this thread] on the MPlayer list:<br />
<br />
==See Also==<br />
[[FFmpeg Wishlist]] for more tasks or ideas.<br />
<br />
[http://bugzilla.mplayerhq.hu/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=FFmpeg&content= FFmpeg bugs] for ffmpeg bugs.</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=User:Gatoatigrado&diff=7340User:Gatoatigrado2007-03-17T21:56:41Z<p>Gatoatigrado: </p>
<hr />
<div>to avoid excess repetitive user pages, please go to my website [http://ntung.com ntung.com].</div>Gatoatigradohttps://wiki.multimedia.cx/index.php?title=Talk:FFmpeg_Summer_Of_Code_2007&diff=7339Talk:FFmpeg Summer Of Code 20072007-03-17T21:56:06Z<p>Gatoatigrado: /* TIFF/TGA encoder? */</p>
<hr />
<div>== TIFF/TGA encoder? ==<br />
<br />
Why? It seems too small a task to be done in the course of SoC. --[[User:Kostya|Kostya]] 09:20, 7 March 2007 (EST)<br />
<br />
: True, I'll rework it. --[[User:Merbanan|Merbanan]] 11:44, 7 March 2007 (EST)<br />
<br />
: So now it's a qualification task? Is this doable in 40+ hours? I don't think I'll be able to spend much more time until the applications are due. --[[User:Gatoatigrado|Gatoatigrado]] 17:56, 17 March 2007 (EDT)<br />
<br />
== DPX Encoder/Decoder ==<br />
From what I understand this is the output of most film scanners. Having not used one I'm not 100% sure of this, and maybe someone could clarify, but my understanding is that film scanners output a collection of these files (I'm unsure if they're put in a container or not). There are already open source implementations in imagemagick and graphics magick.<br />
<br />
: added some info and links - [[User:Compn|Compn]] 20:25, 8 March 2007 (EST)<br />
<br />
== ATRAC3 ==<br />
Should http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/44526 be added to small tasks?<br />
[[User:Ce|Ce]] 04:41, 8 March 2007 (EST)<br />
<br />
: No, I'll fix it eventually, it's not that simple.--[[User:Merbanan|Merbanan]] 06:46, 8 March 2007 (EST)<br />
<br />
== Qualification Tasks ==<br />
is reviewing and fixing up old unapplied ffmpeg patches not a good small task? [[User:Compn|Compn]] 20:25, 8 March 2007 (EST)<br />
<br />
:I wanted to ask the same question a minute ago;-) The QTRLE patch is not ready yet, and there is no patch for an DNxHD ENcoder in the mailing list. I originally wrote that someone with better knowledge should of course remove those proposals if he thinks they are bad, but I still think they probably tell something about applicants (and that Baptiste possibly didn't read Merbanans "rules"). --[[User:Ce|Ce]] 20:31, 8 March 2007 (EST)<br />
<br />
== Photo Codecs ==<br />
<br />
I see several GSoC project proposals for encoding and decoding a variety of images. Is this truly appropriate for the FFmpeg project, which traditionally focuses on sequences of moving pictures vs. single images? I know that FFmpeg can put a movie together from a sequence of still pictures, or dump a movie into a series of still pictures. Are we hoping to do the same with HDR images? Otherwise, this type of work seems best left to dedicated photo processing projects. --[[User:Multimedia Mike|Multimedia Mike]] 13:36, 9 March 2007 (EST)<br />
<br />
: And they are designed to operate with colour formats not currently supported by FFmpeg (like 16 bit per component). But JPEG-2000 and HD-Photo are likely to be used in movies so their support is undoubtedly useful. BTW, how do you plan to use qualification projects? --[[User:Kostya|Kostya]] 14:09, 9 March 2007 (EST)<br />
<br />
::I'm supportive of something like JPEG-2000 (which uses a colorspace FFmpeg does not presently support, IIRC) since I know that there are plans to include that with certain types of movies. I'm not as enthusiastic about graphic formats that are not known to be encoded as sequences of images in a video file. As for the qualifier projects, we are hoping to weed out unqualified applicants by asking that they perform a task from the list. --[[User:Multimedia Mike|Multimedia Mike]] 16:02, 9 March 2007 (EST)<br />
<br />
: i was thinking of that exact feature (convert dpx/tif/exr to h264). also i wonder if the encoder feature could be used for filmmaking using ffmpeg ? e.g. grab from camera right to HD image format? or is this too high end/specialized/low user count? --[[User:Compn|Compn]] 18:52, 9 March 2007 (EST)<br />
<br />
:: You know me-- low user count is not a legit reason for discounting a feature. :-) If there is a legitimate video-type app for a certain feature I think that makes it more relevant to FFmpeg. --[[User:Multimedia Mike|Multimedia Mike]] 19:39, 9 March 2007 (EST)<br />
<br />
::: btw i dont think they need to be SOC projects. exr and dpx are open and have gpl libs (dunno about microsoft hd photo). maybe just a qualifying task or just a wishlist. --[[User:Compn|Compn]] 22:53, 9 March 2007 (EST)<br />
<br />
== AAC ==<br />
<br />
I think finishing AAC support would be a good SoC project. The decoder started in 2006 is far from finished and the project appears to be quite complex. Baptiste appears to disagree, can we come to a consensus here? Does anybody know the exact status of the AAC implementation from 2006? -- [[User:DonDiego|DonDiego]] 10:11, 12 March 2007 (EDT)<br />
<br />
: I think the LC part is almost complete. Adding He-AAC(+) features could be a SoC task.--[[User:Merbanan|Merbanan]] 17:05, 12 March 2007 (EDT)<br />
<br />
==Dirac==<br />
is dirac even finished spec wise? i'd rather see a decoder for the files in the wild (eac3, aac, gsm, 263, indeo)...<br />
<br />
: Their site says it's "essentially complete". Granted, it's not out there in the wild yet, but I think that's mostly a matter of time, so why not get a decoder in now? It's not like libavcodec doesn't already have decoders for a number of other rather obscure formats already. -- [[User:Koorogi|Koorogi]] 21:58, 14 March 2007 (EDT)<br />
<br />
== VP6 Encoder ==<br />
I'm currently writing a VP6 encoder. This project wouldn't be a complete duplication of effort (since mine won't be part of libavcodec), but just so you know. --[[User:Pengvado|Pengvado]] 17:04, 15 March 2007 (EDT)<br />
<br />
: Why can't you integrate it into libavcodec ? It would be more convenient. What license will it have, GPL ? --[[User:Bcoudurier|Bcoudurier]] 09:07, 16 March 2007 (EDT)<br />
<br />
== JPEG2000 ==<br />
Do we have a mentor yet ? We is able to mentor that (wavelet codec) ? Loren ? --[[User:Bcoudurier|Bcoudurier]] 12:28, 16 March 2007 (EDT)<br />
: I filed an application and if it will be accepted any mentor would do. --[[User:Kostya|Kostya]] 15:24, 16 March 2007 (EDT)</div>Gatoatigrado