MultimediaWiki - User contributions [en]

FFmpeg / Libav Summer Of Code

2009-04-24T15:05:29Z

Koorogi: link to MPEG ALS page

The [[FFmpeg]] project has been a participant in the [http://code.google.com/soc/ Google Summer of Code] program since 2006.

* [[FFmpeg_Summer_Of_Code_2009|2009 project page]]
* [[FFmpeg Summer Of Code 2008|2008 project page]]
* [[FFmpeg Summer Of Code 2007|2007 project page]]
* [[FFmpeg Summer Of Code 2006|2006 project page]]

Each accepted project is developed in its own sandbox, separate from the main FFmpeg codebase. Naturally, the end goal of each of the accepted FFmpeg projects ought to be to have that code in shape for acceptance into the production codebase. This page tracks the status of each project and how well each student did.

== 2006 Projects ==

=== VC-1 Decoder ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Multimedia Mike|Mike Melanson]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== AMR-NB Decoder ===
* Student: [[User:superdump|Robert Swain]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: FFmpeg committer. Project not finished during SoC.
* Code Status: Floating point code has been implemented up to synthesis. The next step is to debug the synthesis input and code. Documented on [[AMR-NB]].

=== AC-3 Decoder ===
* Student: [[User:Cloud9|Kartikey Mahendra BHATT]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: disappeared, project unfinished
* Code Status: Picked up by [[User:Jruggle|Justin Ruggles]] and committed to FFmpeg.

=== AAC Decoder ===
* Student: Maxim Gavrilov
* Mentor: [[User:ods15|Oded Shimon]]
* Student Status: disappeared, project unfinished
* Code Status: Picked up by [[User:andoma|Andreas Öman]] and [[User:superdump|Robert Swain]] and committed to FFmpeg.

=== Vorbis Encoder ===
* Student: Mathew Philip
* Mentor: [[User:ods15|Oded Shimon]]
* Student Status: disappeared, project barely started
* Code Status: Picked up by [[User:ods15|Oded Shimon]] and committed to FFmpeg.

== 2007 Projects ==

=== RealVideo 4 Decoder ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Multimedia Mike|Mike Melanson]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase. The project goal morphed to include a RealVideo 3 decoder since the 2 schemes are so similar.

=== QCELP Decoder ===
* Student: [[User:Reynaldo|Reynaldo Verdejo Pinochet]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: FFmpeg commiter. Produced a working decoder during SoC but the code didn't reach SVN before the end of the program.
* Code Status: Picked up by Kenan Gillet and with the help of [[User:Reynaldo|Reynaldo]] committed to FFmpeg. Some features still missing, though.

=== Matroska Muxer ===
* Student: David Conrad
* Mentor: [[User:aurel|Aurélien Jacobs]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== Video Filter API (AKA [[Libavfilter|libavfilter]]) ===
* Student: [[User:Koorogi|Bobby Bingham]]
* Mentor: [[User:Merbanan|Benjamin Larsson]] and Michael Niedermayer
* Student Status: project not finished during SoC but continues working on it
* Code Status: Parts have been committed to FFmpeg, but remain disabled. Still in development (albeit slowly) by [[User:Koorogi|Bobby Bingham]] and [[User:Vitor|Vitor]]. 2009 SoC projects are underway to complete its integration and add audio support.

=== E-AC-3 Decoder ===
* Student: Bartlomiej Wolowiec
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: FFmpeg committer; code produced worked for most available samples, but there were some unimplemented features.
* Code Status: Picked up by [[User:Jruggle|Justin Ruggles]], finished and committed to FFmpeg.

=== JPEG 2000 Encoder and Decoder ===
* Student: Kamil Nowosad
* Mentor: [[User:pengvado|Loren Merritt]]
* Student Status: disappeared, project unfinished
* Code Status: The code is working but not all features are supported.

=== Dirac Encoder and Decoder ===
* Student: Marco Gerards
* Mentor: [[User:Lu_zero|Luca Barbato]]
* Student Status: Project not finished during SoC but continues working on it.
* Code Status: The decoder is in good shape, the encoder still needs more work. Both need to be updated to the latest spec.

=== TS Muxer ===
* Student: Xiaohui Sun
* Mentor: [[User:bcoudurier|Baptiste Coudurier]]
* Student Status: disappeared, project unfinished
* Code Status: [[Interesting Patches#PES packetizer by Xiaohui Sun|Changes]] requested during the review process for FFmpeg inclusion were never made.

== 2008 Projects ==

=== Generic frame-level multithreading support ===
* Student: Alexander Strange
* Mentor: Kristian Jerpetjoen
* Student Status: active
* Code Status: in process here: http://gitorious.org/projects/ffmpeg/repos/ffmpeg-mt

=== Nellymoser Encoder ===
* Student: Bartlomiej Wolowiec
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== ALAC Encoder ===
* Student: [[User:Jai|Jai Menon]]
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== LGPL reimplementation of GPL sws_scale parts ===
* Student: Keiji Costantini
* Mentor: [[User:Lu_zero|Luca Barbato]]
* Student Status: project unfinished
* Code Status: The GPL YUV table generator has since been implemented as LGPL by [[User:Kostya|Kostya Shishkov]].

=== AAC-LC Encoder ===
* Student: [[User:Kostya|Kostya]]
* Mentor: [[User:Andoma|Andreas Öman]]
* Student Status: active
* Code Status: in process

=== MLP/TrueHD encoder ===
* Student: [[User:Angustia|Ramiro Polla]]
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: active
* Code Status: in process, the first stage, getting the MLP decoder into shape and committed, has been accomplished.

=== WMA Pro Decoder ===
* Student: Sascha Sommer
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: active
* Code Status: in process, fully functional code in SoC SVN tree [http://svn.ffmpeg.org/soc/wmapro/]

=== MXF Muxer ===
* Student: [[User:spyfeng|Zhentan Feng]]
* Mentor: [[User:Bcoudurier|Baptiste Coudurier]]
* Student Status: finished project
* Code Status: Accepted into the FFmpeg codebase.

== 2009 Projects ==

=== RTMP Support (Flash streaming) ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Ronald S. Bultje|Ronald Bultje]]
* Student Status: active
* Code Status: in process

=== libswscale cleanup ===
* Student: [[User:Angustia|Ramiro Polla]]
* Mentor: [[User:reimar|Reimar Döffinger]]
* Student Status: active
* Code Status: in process

=== S/PDIF muxer ===
* Student: Bartlomiej Wolowiec
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: active
* Code Status: in process

=== Playlist/Concatenation Support for FFmpeg ===
* Student: Geza Kovacs
* Mentor: [[User:Bcoudurier|Baptiste Coudurier]]
* Student Status: active
* Code Status: in process

=== JPEG2000 decoder and encoder ===
* Student: [[User:Jai|Jai Menon]]
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: active
* Code Status: in process

=== Implement the New Seeking API in Libavformat ===
* Student: [[User:spyfeng|Zhentan Feng]]
* Mentor: [[User:Bcoudurier|Baptiste Coudurier]]
* Student Status: active
* Code Status: in process

=== [[MPEG-4 Audio Lossless Coding (ALS)|MPEG-4 ALS]] decoder ===
* Student: Thilo Borgmann
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: active
* Code Status: in process

=== Implementation of AVFilter infrastructure and various audio filters ===
* Student: Kevin Dubois
* Mentor: [[User:Vitor|Vitor Sessak]]
* Student Status: active
* Code Status: in process

=== Finish AMR-NB decoder and write an encoder ===
* Student: Colin McQuillan
* Mentor: [[User:superdump|Robert Swain]]
* Student Status: active
* Code Status: in process

[[Category:FFmpeg]]

FFmpeg / Libav Summer Of Code

2009-04-20T22:50:49Z

Koorogi: /* Video Filter API (AKA libavfilter) */

The [[FFmpeg]] project has been a participant in the [http://code.google.com/soc/ Google Summer of Code] program since 2006.

* [[FFmpeg_Summer_Of_Code_2009|2009 project page]]
* [[FFmpeg Summer Of Code 2008|2008 project page]]
* [[FFmpeg Summer Of Code 2007|2007 project page]]
* [[FFmpeg Summer Of Code 2006|2006 project page]]

Each accepted project is developed in its own sandbox, separate from the main FFmpeg codebase. Naturally, the end goal of each of the accepted FFmpeg projects ought to be to have that code in shape for acceptance into the production codebase. This page tracks the status of each project and how well each student did.

== 2006 Projects ==

=== VC-1 Decoder ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Multimedia Mike|Mike Melanson]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== AMR-NB Decoder ===
* Student: [[User:superdump|Robert Swain]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: FFmpeg committer. Project not finished during SoC.
* Code Status: Floating point code has been implemented up to synthesis. The next step is to debug the synthesis input and code. Documented on [[AMR-NB]].

=== AC-3 Decoder ===
* Student: [[User:Cloud9|Kartikey Mahendra BHATT]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: disappeared, project unfinished
* Code Status: Picked up by [[User:Jruggle|Justin Ruggles]] and committed to FFmpeg.

=== AAC Decoder ===
* Student: Maxim Gavrilov
* Mentor: [[User:ods15|Oded Shimon]]
* Student Status: disappeared, project unfinished
* Code Status: Picked up by [[User:andoma|Andreas Öman]] and [[User:superdump|Robert Swain]] and committed to FFmpeg.

=== Vorbis Encoder ===
* Student: Mathew Philip
* Mentor: [[User:ods15|Oded Shimon]]
* Student Status: disappeared, project barely started
* Code Status: Picked up by [[User:ods15|Oded Shimon]] and committed to FFmpeg.

== 2007 Projects ==

=== RealVideo 4 Decoder ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Multimedia Mike|Mike Melanson]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase. The project goal morphed to include a RealVideo 3 decoder since the 2 schemes are so similar.

=== QCELP Decoder ===
* Student: [[User:Reynaldo|Reynaldo Verdejo Pinochet]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: FFmpeg commiter. Produced a working decoder during SoC but the code didn't reach SVN before the end of the program.
* Code Status: Picked up by Kenan Gillet and with the help of [[User:Reynaldo|Reynaldo]] committed to FFmpeg. Some features still missing, though.

=== Matroska Muxer ===
* Student: David Conrad
* Mentor: [[User:aurel|Aurélien Jacobs]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== Video Filter API (AKA [[Libavfilter|libavfilter]]) ===
* Student: [[User:Koorogi|Bobby Bingham]]
* Mentor: [[User:Merbanan|Benjamin Larsson]] and Michael Niedermayer
* Student Status: project not finished during SoC but continues working on it
* Code Status: Parts have been committed to FFmpeg, but remain disabled. Still in development (albeit slowly) by [[User:Koorogi|Bobby Bingham]] and [[User:Vitor|Vitor]]. 2009 SoC projects are underway to complete its integration and add audio support.

=== E-AC-3 Decoder ===
* Student: Bartlomiej Wolowiec
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: FFmpeg committer; code produced worked for most available samples, but there were some unimplemented features.
* Code Status: Picked up by [[User:Jruggle|Justin Ruggles]], finished and committed to FFmpeg.

=== JPEG 2000 Encoder and Decoder ===
* Student: Kamil Nowosad
* Mentor: [[User:pengvado|Loren Merritt]]
* Student Status: disappeared, project unfinished
* Code Status: The code is working but not all features are supported.

=== Dirac Encoder and Decoder ===
* Student: Marco Gerards
* Mentor: [[User:Lu_zero|Luca Barbato]]
* Student Status: Project not finished during SoC but continues working on it.
* Code Status: The decoder is in good shape, the encoder still needs more work. Both need to be updated to the latest spec.

=== TS Muxer ===
* Student: Xiaohui Sun
* Mentor: [[User:bcoudurier|Baptiste Coudurier]]
* Student Status: disappeared, project unfinished
* Code Status: [[Interesting Patches#PES packetizer by Xiaohui Sun|Changes]] requested during the review process for FFmpeg inclusion were never made.

== 2008 Projects ==

=== Generic frame-level multithreading support ===
* Student: Alexander Strange
* Mentor: Kristian Jerpetjoen
* Student Status: active
* Code Status: in process here: http://gitorious.org/projects/ffmpeg/repos/ffmpeg-mt

=== Nellymoser Encoder ===
* Student: Bartlomiej Wolowiec
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== ALAC Encoder ===
* Student: [[User:Jai|Jai Menon]]
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== LGPL reimplementation of GPL sws_scale parts ===
* Student: Keiji Costantini
* Mentor: [[User:Lu_zero|Luca Barbato]]
* Student Status: project unfinished
* Code Status: The GPL YUV table generator has since been implemented as LGPL by [[User:Kostya|Kostya Shishkov]].

=== AAC-LC Encoder ===
* Student: [[User:Kostya|Kostya]]
* Mentor: [[User:Andoma|Andreas Öman]]
* Student Status: active
* Code Status: in process

=== MLP/TrueHD encoder ===
* Student: [[User:Angustia|Ramiro Polla]]
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: active
* Code Status: in process, the first stage, getting the MLP decoder into shape and committed, has been accomplished.

=== WMA Pro Decoder ===
* Student: Sascha Sommer
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: active
* Code Status: in process, fully functional code in SoC SVN tree [http://svn.ffmpeg.org/soc/wmapro/]

=== MXF Muxer ===
* Student: [[User:spyfeng|Zhentan Feng]]
* Mentor: [[User:Bcoudurier|Baptiste Coudurier]]
* Student Status: finished project
* Code Status: Accepted into the FFmpeg codebase.

== 2009 Projects ==

=== RTMP Support (Flash streaming) ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Ronald S. Bultje|Ronald Bultje]]
* Student Status: active
* Code Status: in process

=== Finish libavfilter integration ===
* Student: [[User:Angustia|Ramiro Polla]]
* Mentor: [[User:reimar|Reimar Döffinger]]
* Student Status: active
* Code Status: in process

=== S/PDIF muxer ===
* Student: Bartlomiej Wolowiec
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: active
* Code Status: in process

=== Playlist/Concatenation Support for FFmpeg ===
* Student: Geza Kovacs
* Mentor: [[User:Bcoudurier|Baptiste Coudurier]]
* Student Status: active
* Code Status: in process

=== JPEG2000 decoder and encoder ===
* Student: [[User:Jai|Jai Menon]]
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: active
* Code Status: in process

=== Implement the New Seeking API in Libavformat ===
* Student: [[User:spyfeng|Zhentan Feng]]
* Mentor: [[User:Bcoudurier|Baptiste Coudurier]]
* Student Status: active
* Code Status: in process

=== MPEG-4 ALS decoder ===
* Student: Thilo Borgmann
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: active
* Code Status: in process

=== Implementation of AVFilter infrastructure and various audio filters ===
* Student: Kevin Dubois
* Mentor: [[User:Vitor|Vitor Sessak]]
* Student Status: active
* Code Status: in process

=== Finish AMR-NB decoder and write an encoder ===
* Student: Colin McQuillan
* Mentor: [[User:superdump|Robert Swain]]
* Student Status: active
* Code Status: in process

[[Category:FFmpeg]]

Talk:FFmpeg Summer Of Code 2009

2009-01-17T21:26:44Z

Koorogi: /* libavui (a common skins library)? */

=== S/PDIF muxer ===

Is there any specific qualification task you would like done for this? -- Jai

:Working Jpeg2000 decoder ;), cleaning up this http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2005-June/001673.html would be welcome. It's a rpza encoder. --[[User:Merbanan|Merbanan]] 06:22, 31 December 2008 (EST)

=== speex + gsm ===

Aren't libgsm and libspeex distributed under a permissive license?
If yes, these tasks do not have very high priority, imo.
[[User:Ce|Ce]] 14:56, 11 January 2009 (EST)

=== DTS-HD Master Audio decoder? ===
Would [http://en.wikipedia.org/wiki/DTS-HD_Master_Audio DTS-HD Master Audio] decoder make good project suggestion? [[User:Gamester17|Gamester17]] 02:51, 16 January 2009 (EST)

http://en.wikipedia.org/wiki/DTS-HD_Master_Audio
:""''DTS-HD Master Audio is a lossless audio codec created by Digital Theater System. It was previously known as DTS++ and DTS-HD. It is an extension of DTS which, when played back on devices which do not support the Master Audio extension, degrades to a 1.5 Mbit/s "core" track which is lossy. DTS-HD Master Audio is an optional audio format for both Blu-ray Disc and HD DVD''"

Specs, please. From what I know the projects without spec take a looong time to complete. --[[User:Kostya|Kostya]] 03:32, 16 January 2009 (EST)

:AFAIK, there is even no software implementation, so it would be even more difficult;-( [[User:Ce|Ce]] 20:14, 16 January 2009 (EST)

=== WTV (Microsoft Windows Media Center Recording Format) demuxer? ===
Would a [[WTV|WTV (Microsoft Windows Media Center Recording Format)]] demuxer make good project suggestion? [[User:Gamester17|Gamester17]] 13:14, 16 January 2009 (EST)

[[WTV]]
:"''WTV is the new container format used to record television shows in Microsoft Windows Vista Media Center starting with Windows Media Center TV Pack 2008.''", "''WTV is the successor of DVR-MS which is is being replaced with WTV''", "''WRT is also the default recording format for Windows 7 Media Center''"

::This is tricky. It doesn't strike me as being involved enough to qualify as one of our usual SoC projects. OTOH, it seems a little too involved to be a qualification task. --[[User:Multimedia Mike|Multimedia Mike]] 14:24, 16 January 2009 (EST)

=== libavui (a common skins library)? ===
Would a common skins library make good project suggestion?
*MPlayer skin
*VLC skin
*Xine skin
*XMMS skin
*WINAMP skin
*Windows Media Player skin
-[[User:Nazo|Nazo]] 21:29, 16 January 2009 (EST)
:: Personally, I would advocate a project to stamp out skinnable UIs across the computing landscape. But that's outside of the scope of an SoC project. I hate UI skins. --[[User:Multimedia Mike|Multimedia Mike]] 14:03, 17 January 2009 (EST)
:::: I second that. But I don't see how GUI stuff like promoting or discouraging skins relates to libav* in the first place. [[User:Koorogi|Koorogi]] 16:26, 17 January 2009 (EST)

Talk:TX2

2008-01-29T17:07:09Z

Koorogi: request clarification on swizzling

Can somebody offer any clarification on when the image data is swizzled, and when not? The article just saying "sometimes" isn't very helpful. I'll take a look when I get a chance, but it might be a while, so if somebody happens to know, or has some free time to spend on it, a little clarification would be nice. --[[User:Koorogi|Koorogi]] 12:07, 29 January 2008 (EST)

FFmpeg filter HOWTO

2007-12-23T23:52:15Z

Koorogi: document the new API since the great colorspace API change

This page is meant as an introduction of writing filters for [[libavfilter]]. This is a work in progress, but should at least point you in the right direction for writing simple filters.

== Definition of a filter ==

=== AVFilter ===
All filters are described by an AVFilter structure. This structure gives information needed to initialize the filter, and information on the entry points into the filter code. This structure is declared in libavfilter/avfilter.h:

typedef struct
{
char *name; ///< filter name
char *author; ///< filter author

int priv_size; ///< size of private data to allocate for the filter

int (*init)(AVFilterContext *ctx, const char *args, void *opaque);
void (*uninit)(AVFilterContext *ctx);

int (*query_formats)(AVFilterContext *ctx);

const AVFilterPad *inputs; ///< NULL terminated list of inputs. NULL if none
const AVFilterPad *outputs; ///< NULL terminated list of outputs. NULL if none
} AVFilter;

The query_formats function sets the in_formats member of connected '''output''' links, and the out_formats member of connected '''input''' links, described below under AVFilterLink.

=== AVFilterPad ===
Let's take a quick look at the AVFilterPad structure, which is used to describe the inputs and outputs of the filter. This is also defined in libavfilter/avfilter.h:

typedef struct AVFilterPad
{
char *name;
int type;

int min_perms;
int rej_perms;

void (*start_frame)(AVFilterLink *link, AVFilterPicRef *picref);
AVFilterPicRef *(*get_video_buffer)(AVFilterLink *link, int perms);
void (*end_frame)(AVFilterLink *link);
void (*draw_slice)(AVFilterLink *link, int y, int height);

int (*request_frame)(AVFilterLink *link);

int (*config_props)(AVFilterLink *link);
} AVFilterPad;

The actual definition in the header file has doxygen comments describing each entry point, its purpose, and what type of pads it is relevant for. These fields are relevant for all pads:

{| border="1"
!Field
!Description
|-
|name
|Name of the pad. No two inputs should have the same name, and no two outputs should have the same name.
|-
|type
|Only AV_PAD_VIDEO currently.
|-
|config_props
|Handles configuration of the link connected to the pad
|}

Fields only relevant to input pads are:

{| border="1"
!Field
!Description
|-
|min_perms
|Minimum permissions required to a picture received as input.
|-
|rej_perms
|Permissions not accepted on pictures received as input.
|-
|start_frame
|Called when a frame is about to be given as input.
|-
|draw_slice
|Called when a slice of frame data has been given as input.
|-
|end_frame
|Called when the input frame has been completely sent.
|-
|get_video_buffer
|Called by the previous filter to request memory for a picture.
|}

Fields only relevant to output pads are:

{| border="1"
!Field
!Description
|-
|request_frame
|Requests that the filter output a frame.
|}

== Picture buffers ==

=== Reference counting ===
All pictures in the filter system are reference counted. This means that there is a picture buffer with memory allocated for the image data, and various filters can own a reference to the buffer. When a reference is no longer needed, its owner frees the reference. When the last reference to a picture buffer is freed, the filter system automatically frees the picture buffer.

=== Permissions ===
The upshot of multiple filters having references to a single picture is that they will all want some level of access to the image data. It should be obvious that if one filter expects to be able to read the image data without it changing that no other filter should write to the image data. The permissions system handles this.

In most cases, when a filter prepares to output a frame, it will request a buffer from the filter to which it will be outputting. It specifies the minimum permissions it needs to the buffer, though it may be given a buffer with more permissions than the minimum it requested.

When it wants to pass this buffer to another filter as output, it creates a new reference to the picture, possibly with a reduced set of permissions. This new reference will be owned by the filter receiving it.

So, for example, for a filter which drops frames if they are similar to the last frame it output, it would want to keep its own reference to a picture after outputting it, and make sure that no other filter modified the buffer either. It would do this by requesting the permissions AV_PERM_READ|AV_PERM_WRITE|AV_PERM_PRESERVE for itself, and removing the AV_PERM_WRITE permission from any references it gave to other filters.

The available permissions are:
{| border="1"
!Permission
!Description
|-
|AV_PERM_READ
|Can read the image data.
|-
|AV_PERM_WRITE
|Can write to the image data.
|-
|AV_PERM_PRESERVE
|Can assume that the image data will not be modified by other filters. This means that no other filters should have the AV_PERM_WRITE permission.
|-
|AV_PERM_REUSE
|The filter may output the same buffer multiple times, but the image data may not be changed for the different outputs.
|-
|AV_PERM_REUSE2
|The filter may output the same buffer multiple times, and may modify the image data between outputs.
|}

== Filter Links ==
A filter's inputs and outputs are connected to those of another filter through the AVFilterLink structure:

typedef struct AVFilterLink
{
AVFilterContext *src; ///< source filter
unsigned int srcpad; ///< index of the output pad on the source filter

AVFilterContext *dst; ///< dest filter
unsigned int dstpad; ///< index of the input pad on the dest filter

int w; ///< agreed upon image width
int h; ///< agreed upon image height
enum PixelFormat format; ///< agreed upon image colorspace

AVFilterFormats *in_formats; ///< formats supported by source filter
AVFilterFormats *out_formats; ///< formats supported by destination filter

AVFilterPicRef *srcpic;

AVFilterPicRef *cur_pic;
AVFilterPicRef *outpic;
};

The src and dst members indicate the filters at the source and destination ends of the link, respectively. The srcpad indicates the index of the output pad on the source filter to which the link is connected. Likewise, the dstpad indicates the index of the input pad on the destination filter.

The in_formats member points to a list of formats supported by the source filter, while the out_formats member points to a list of formats supported by the destination filter. The AVFilterFormats structure used to store the lists is reference counted, and in fact tracks its references (see the comments for the AVFilterFormats structure in libavfilter/avfilter.h for more information on how the colorspace negotiation is works and why this is necessary). The upshot is that if a filter provides pointers to the same list on multiple input/output links, it means that those links will be forced to use the same format as each other.

When two filters are connected, they need to agree upon the dimensions of the image data they'll be working with, and the format that data is in. Once this has been agreed upon, these parameters are stored in the link structure.

The srcpic member is used internally by the filter system, and should not be accessed directly.

The cur_pic member is for the use of the destination filter. When a frame is currently being sent over the link (ie. starting from the call to start_frame() and ending with the call to end_frame()), this contains the reference to the frame which is owned by the destination filter.

The outpic member is described in the following tutorial on writing a simple filter.

== Writing a simple filter ==

=== Default filter entry points ===
Because the majority of filters that will probably be written will take exactly one input, and produce exactly one output, and output one frame for every frame received as input, the filter system provides a number default entry points to ease the development of such filters.

{| border="1"
!Entry point
!Actions taken by the default implementation
|-
|request_frame()
|Request a frame from the previous filter in the chain.
|-
|query_formats()
|Sets the list of supported formats on all input pads such that all links must use the same format, from a default list of formats containing most YUV and RGB/BGR formats.
|-
|start_frame()
|Request a buffer to store the output frame in. A reference to this buffer is stored in the outpic member of the link hooked to the filter's output. The next filter's start_frame() callback is called and given a reference to this buffer.
|-
|end_frame()
|Calls the next filter's end_frame() callback. Frees the reference to the outpic member of the output link, if it was set by (ie. if the default start_frame() is used). Frees the cur_pic reference in the input link.
|-
|get_video_buffer()
|Returns a buffer with the AV_PERM_READ permission in addition to all the requested permissions.
|-
|config_props() on output pad
|Sets the image dimensions for the output link to the same as on the filter's input.
|}

=== The vf_negate filter ===
Having looked at the data structures and callback functions involved, let's take a look at an actual filter. The vf_negate filter inverts the colors in a video. It has one input, and one output, and outputs exactly one frame for every input frame. In this way, it's fairly typical, and can take advantage of many of the default callback implementations offered by the filter system.

First, let's take a look at the AVFilter structure at the bottom of the libavfilter/vf_negate.c file:

AVFilter avfilter_vf_negate =
{
.name = "negate",
.author = "Bobby Bingham",

.priv_size = sizeof(NegContext),

.query_formats = query_formats,

.inputs = (AVFilterPad[]) {{ .name = "default",
.type = AV_PAD_VIDEO,
.draw_slice = draw_slice,
.config_props = config_props,
.min_perms = AV_PERM_READ, },
{ .name = NULL}},
.outputs = (AVFilterPad[]) {{ .name = "default",
.type = AV_PAD_VIDEO, },
{ .name = NULL}},
};

Here, you can see that the filter is named "negate," and it needs sizeof(NegContext) bytes of data to store its context. In the list of inputs and outputs, a pad whose name is set to NULL indicates the end of the list, so this filter has exactly one input and one output. If you look closely at the pad definitions, you will see that fairly few callback functions are actually specified. Because of the simplicity of the filter, the defaults can do most of the work for us.

Let us take a look at the callback function it does define.

==== query_formats() ====
static int query_formats(AVFilterContext *ctx)
{
avfilter_set_common_formats(ctx,
avfilter_make_format_list(10,
PIX_FMT_YUV444P, PIX_FMT_YUV422P, PIX_FMT_YUV420P,
PIX_FMT_YUV411P, PIX_FMT_YUV410P,
PIX_FMT_YUVJ444P, PIX_FMT_YUVJ422P, PIX_FMT_YUVJ420P,
PIX_FMT_YUV440P, PIX_FMT_YUVJ440P));
return 0;
}

This calls avfilter_make_format_list(). This function takes as its first parameter the number of formats which will follow as the remaining parameters. The return value is an AVFilterFormats structure containing the given formats. The avfilter_set_common_formats() function which this structure is passed to sets all connected links to use this same list of formats, which causes all the filters to use the same format after negotiation is complete. As you can see, this filter supports a number of planar YUV colorspaces, including JPEG YUV colorspaces (the ones with a 'J' in the names).

==== config_props() on an input pad ====
The config_props() on an input pad is responsible for verifying that the properties of the input pad are supported by the filter, and to make any updates to the filter's context which are necessary for the link's properties.

TODO: quick explanation of YUV colorspaces, chroma subsampling, difference in range of YUV and JPEG YUV.

Let's take a look at the way in which this filter stores its context:

typedef struct
{
int offY, offUV;
int hsub, vsub;
} NegContext;

That's right. The priv_size member of the AVFilter structure tells the filter system how many bytes to reserve for this structure. The hsub and vsub members are used for chroma subsampling, and the offY and offUV members are used for handling the difference in range between YUV and JPEG YUV. Let's see how these are set in the input pad's config_props:

static int config_props(AVFilterLink *link)
{
NegContext *neg = link->dst->priv;

avcodec_get_chroma_sub_sample(link->format, &neg->hsub, &neg->vsub);

switch(link->format) {
case PIX_FMT_YUVJ444P:
case PIX_FMT_YUVJ422P:
case PIX_FMT_YUVJ420P:
case PIX_FMT_YUVJ440P:
neg->offY =
neg->offUV = 0;
break;
default:
neg->offY = -4;
neg->offUV = 1;
}

return 0;
}

This simply calls avcodec_get_chroma_sub_sample() to get the chroma subsampling shift factors, and stores those in the context. It then stores a set of offsets for compensating for different luma/chroma value ranges for JPEG YUV, and a different set of offsets for other YUV colorspaces. It returns zero to indicate success, because there are no possible input cases which this filter cannot handle.

==== draw_slice() ====
Finally, the function which actually does the processing for the filter, draw_slice():

static void draw_slice(AVFilterLink *link, int y, int h)
{
NegContext *neg = link->dst->priv;
AVFilterPicRef *in = link->cur_pic;
AVFilterPicRef *out = link->dst->outputs[0]->outpic;
uint8_t *inrow, *outrow;
int i, j, plane;

/* luma plane */
inrow = in-> data[0] + y * in-> linesize[0];
outrow = out->data[0] + y * out->linesize[0];
for(i = 0; i < h; i ++) {
for(j = 0; j < link->w; j ++)
outrow[j] = 255 - inrow[j] + neg->offY;
inrow += in-> linesize[0];
outrow += out->linesize[0];
}

/* chroma planes */
for(plane = 1; plane < 3; plane ++) {
inrow = in-> data[plane] + (y >> neg->vsub) * in-> linesize[plane];
outrow = out->data[plane] + (y >> neg->vsub) * out->linesize[plane];

for(i = 0; i < h >> neg->vsub; i ++) {
for(j = 0; j < link->w >> neg->hsub; j ++)
outrow[j] = 255 - inrow[j] + neg->offUV;
inrow += in-> linesize[plane];
outrow += out->linesize[plane];
}
}

avfilter_draw_slice(link->dst->outputs[0], y, h);
}

The y parameter indicates the top of the current slice, and the h parameter the slice's height. Areas of the image outside this slice should not be assumed to be meaningful (though a method to allow this assumption in order to simplify boundary cases for some filters is coming in the future).

This sets inrow to point to the beginning of the first row of the slice in the input, and outrow similarly for the output. Then, for each row, it loops through all the pixels, subtracting them from 255, and adding the offset which was determined in config_props() to account for different value ranges.

It then does the same thing for the chroma planes. Note how the width and height are shifted right to account for the chroma subsampling.

Once the drawing is completed, the slice is sent to the next filter by calling avfilter_draw_slice().

[[Category:FFmpeg Tutorials]]
[[Category:FFmpeg]]

FFmpeg Wishlist

2007-12-23T23:09:25Z

Koorogi: h/vflip can now be done with libavfilter

The various desired features for FFmpeg can be found in the following pages:
* [https://roundup.mplayerhq.hu/roundup/ffmpeg/issue?%40&%40columns=title&%40columns=topic&id=&%40columns=id&%40columns=activity&%40sort=activity&%40columns=priority&%40group=priority&type=2&status=2&%40columns=substatus&%40pagesize=50&%40startwith=0&%40action=search Open feature requests in the issue tracker]
* [http://svn.mplayerhq.hu/ffmpeg/trunk/doc/TODO?view=co TODO file in the SVN tree]
* [[FFmpeg Summer Of Code]] pages
* Finish and commit any code not yet commited at the [http://svn.mplayerhq.hu/soc/ SoC FFmpeg tree]

Also, other features requests can be found in:
* [https://roundup.mplayerhq.hu/roundup/ffmpeg/issue?%40&%40columns=title&%40columns=topic&id=&%40columns=id&%40columns=activity&%40sort=activity&%40columns=priority&%40group=priority&type=2&status=1&%40columns=substatus&%40pagesize=50&%40startwith=0&%40action=search Feature requests marked as "new"]
* Below in this page (mostly deprecated items)

See also the discussion about [[Ffmpeg audio api|Audio API]] TODOs.

= Old list =

The following is deprecated, please '''do not''' add new items to this list, use instead the issue tracker. Also, send a message to the mailing list before implementing one of those items. They could be work-in-progress or not wanted anymore.

Moving any of these items to a '''proper''' feature request in the issue tracker is welcome.

== Decoders ==
* Add b-frame support to the ffsvq3 decoder
* integrate [[Speex]] (glue code or native)
* Fix "[rv20 @ 009C8BF0]unknown bit3 set" in [[RV20]] decoder
* [[xeb|XEB]] - the [[RatDVD]] video codec (stored in [[xvo|XVO]] container format)
* VNC decoder, files created by vncrec. Re-use code from [[VMware Video]] decoder http://www.sodan.org/~penny/vncrec/
* Additional game formats support:
** [[VQA]] v3 support, see http://www.gamers.org/pub/idgames2/planetquake/planetcnc/cncdz/
** [[Gremlin Digital Video]]
** [[ARMovie|ARMovie/RPL]]
** [[ESCAPE]]
** [[M95]]

== Demuxers ==
* [[FluxDVD]] / [[RatDVD]] demuxer for [[xvo|XVO]] files (Note! [[RatDVD]] is the predecesor of [[FluxDVD]])

== Muxers ==
* DVB (MPEG-TS) muxer inside DVB containers
** MPEG-1/2 video-streams inside DVB containers
** MPEG-4 ASP video-streams inside DVB containers
** MPEG-4 AVC (H.264) video-streams inside DVB containers
** AC3 audio-streams inside DVB containers
*** Mutiple AC3 audio-streams inside DVB containers
** MP3 audio-streams inside DVB containers
*** Mutiple MP3 audio-streams inside DVB containers
* NSV muxer
* NSA muxer

== Features ==
* Create a new audio API system
* Grabbing from video devices under windows
** Apply [[Interesting Patches#Microsoft Video for Windows capture support|this]] VFW capture patch
** Create a DirectShow patch
* Improved exition documentation and add additional means to document
**Web
**WIKI
**manpage

== Misc ==
* Clean up the h263 rtp [[Interesting Patches#H.263 rtp patch|patch]].

== Streaming Media Network Protocols ==
Streaming Media Network Protocols (client and server-side) improvements/enhancements and related ideas for new features/functions.
* Create a common 'stream demuxer/parser library' for the client-side to receive input streams (and/or API for adding support for additional streaming formats?) - a LGPL'ed sub-library in FFmpeg with all stream demuxers/parsers gathered (similar to the libpostproc and libavutil). Call it "libstream" (or "stream" or whatever). Move FFmpeg's existing stream code there like HTTP and RTSP/RTP. This will help reduce future code replication by sharing common code, thus making it easier to add support for additional streaming formats. All togther making it super easy for audio/video players using FFmpeg to add all-in-one streaming support to their player.
**Maybe use either [http://www.mplayerhq.hu MPlayer]'s "''stream''" library structure, [http://www.live555.com LIVE555], [http://curl.haxx.se cURL], or probebly the better [http://streaming.polito.it/client/library libnms] (from [http://streaming.polito.it/client NeMeSi]) as a base for such a common library?
*Add support for additional streaming protocols (on the client side) and improve/enhance support for existing protocols:
** HTTP (Hypertext Transfer Protocol) client
*** plus a SSL (Secure Sockets Layer) client support for HTTPS
** UDP (User Datagram Protocol) client
** RTSP - Real-Time Streaming Protocol (RFC2326) client
** RTP/RTCP - Real-Time Transport Protocol/RTP Control Protocol (RFC3550) client
** RTP Profile for Audio and Video Conferences with Minimal Control (RFC3551) client
** RealMedia RTSP/RDT (Real Time Streaming Protocol / Real Data Transport) client
** SDP (Service Discovery Protocol) / SSDP (Simple Service Discovery Protocol) client
** MMS (Microsoft Media Services) client
*** including the subprotocol mmsh (MMS over HTTP) and mmst (MMS over TCP)
*FFServer (streaming server) updating and improving:
**FFServer code hasn't been update for quite a while
**Support for RTSP interleaved RTP media
**RTSP over HTTP tunneling
**SLL (Secure Sockets Layer) support
**TLS (Transport Layer Security) support
**SCTP (Stream Control Transmission Protocol) support
***including tunnel SCTP over UDP
**Per-asset accounting options
**Profiling and performance improvements of the RTSP, HTTP and RTP server code
**Streaming to clients like WMP 9, 10 and 11 is broken
**MMS server streaming support in FFServer, (especially for Linux).
*** including the subprotocol mmsh (MMS over HTTP) and mmst (MMS over TCP)
*** Note that al3x has gotten something working with ffserver, you might want to ask him what needs to be done as well :) --[[User:Compn|Compn]] 14:22, 19 March 2007 (EDT)
***You should also take a look at the [http://streaming.polito.it/server FENG (RTSP Streaming Server)] code, [http://streaming.polito.it/embedded NetEmbryo (Embedded Open Media Streaming Library)], and also [http://curl.haxx.se cURL] --[[User:Gamester17|Gamester17]] 11:20, 29 March 2007 (GMT+1)

==See Also==
*[[FFmpeg Summer Of Code]] for more suggestions/requests (ideas for developers).
*[http://bugzilla.mplayerhq.hu/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=FFmpeg&content= FFmpeg bugs] for bugs in FFmpegs (codecs) that you can help fix or add addition information/samples to.
*[[:Category:Formats missing in FFmpeg]] for formats not implemented in ffmpeg yet

[[Category:FFmpeg]]

User:Koorogi

2007-12-23T23:06:10Z

Koorogi: little more info

Bobby Bingham is an [[FFmpeg Summer Of Code 2007]] student working on [[libavfilter]]. He is also semi-fluent in Japanese, and is currently spending his spare time writing a Sega Saturn emulator for the fun of it. Oh, and to learn more about how it all works.

FFmpeg / Libav Summer Of Code

2007-12-23T22:59:28Z

Koorogi: colorspace negotiation has been revamped

The [[FFmpeg]] project has been a participant in the [http://code.google.com/soc/ Google Summer of Code] program during the 2006 and 2007 seaons.

* [[FFmpeg Summer Of Code 2006|2006 project page]]
* [[FFmpeg Summer Of Code 2007|2007 project page]]

Each accepted project is developed in its own sandbox, separate from the main FFmpeg codebase. Naturally, the end goal of each of the accepted FFmpeg projects ought to be to have that code in shape for acceptance into the production codebase. This page tracks the status of each project and how well each student did.

== 2006 Projects ==

=== VC-1 Decoder ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Multimedia Mike|Mike Melanson]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== AMR Decoder ===
* Student: [[User:superdump|Robert Swain]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: project not finished during SoC but continues working on it
* Code Status: To be expected soon. (Within a few weeks from 20071203 I think.) Narrow band decoding documented on [[AMR-NB]] and floating point code has been implemented up to synthesis.

=== AC3 Decoder ===
* Student: [[User:Cloud9|Kartikey Mahendra BHATT]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: disappeared, project unfinished
* Code Status: Picked up by [[User:Jruggle|Justin Ruggles]] and committed to FFmpeg.

=== AAC Decoder ===
* Student: Maxim Gavrilov
* Mentor: [[User:ods15|Oded Shimon]]
* Student Status: disappeared, project unfinished
* Code Status: Picked up by [[User:andoma|Andreas Öman]] who is currently preparing code for merge with FFmpeg.

=== Vorbis Encoder ===
* Student: Mathew Philip
* Mentor: [[User:ods15|Oded Shimon]]
* Student Status: disappeared, project barely started
* Code Status: Picked up by [[User:ods15|Oded Shimon]] and committed to FFmpeg.

== 2007 Projects ==

=== RealVideo 4 Decoder ===
* Student: [[User:Kostya|Kostya Shishkov]]
* Mentor: [[User:Multimedia Mike|Mike Melanson]]
* Student Status: FFmpeg committer
* Code Status: in the process of being committed to FFmpeg; the project goal has also morphed to include a RealVideo 3 decoder since the 2 schemes are so similar. Both RV30 and RV40 are decodeable with visual artifacts.

=== QCELP Decoder ===
* Student: [[User:Reynaldo|Reynaldo Verdejo Pinochet]]
* Mentor: [[User:Merbanan|Benjamin Larsson]]
* Student Status: project not finished during SoC but continues working on it
* Code Status: slowly progressing, it's working though

=== Matroska Muxer ===
* Student: David Conrad
* Mentor: [[User:aurel|Aurélien Jacobs]]
* Student Status: FFmpeg committer
* Code Status: Accepted into the FFmpeg codebase.

=== Video Filter API (AKA [[Libavfilter|libavfilter]]) ===
* Student: [[User:Koorogi|Bobby Bingham]]
* Mentor: [[User:Merbanan|Benjamin Larsson]] and Michael Niedermayer
* Student Status: project not finished during SoC but continues working on it
* Code Status: Working code for ffplay and ffmpeg. Still in development (albeit slowly) by [[User:Koorogi|Bobby Bingham]] and [[User:Vitor|Vitor]].

=== E-AC3 Decoder ===
* Student: Bartlomiej Wolowiec
* Mentor: [[User:Jruggle|Justin Ruggles]]
* Student Status: project not finished during SoC, (continues working on it?)
* Code Status: Working for most available samples. There are still some unimplemented features though. The code is currently not clean enough for inclusion in FFmpeg. Now picked up by [[User:Jruggle|Justin Ruggles]] and being beaten into shape.

=== JPEG 2000 Encoder and Decoder ===
* Student: Kamil Nowosad
* Mentor: [[User:pengvado|Loren Merritt]]
* Student Status: disappeared, project unfinished
* Code Status: The code is working but not all features are supported.

=== Dirac Encoder and Decoder ===
* Student: Marco Gerards
* Mentor: [[User:Lu_zero|Luca Barbato]]
* Student Status: project not finished during SoC but continues working on it, just slower than before due to other tasks taking priority. (Winter vacations approaching!)
* Code Status: The decoder is in good shape, the encoder still needs more work. Both need to be updated to the latest spec.

=== TS Muxer ===
* Student: Xiaohui Sun
* Mentor: [[User:bcoudurier|Baptiste Coudurier]]
* Student Status: disappeared, project unfinished
* Code Status: [[Interesting Patches#PES packetizer by Xiaohui Sun|Changes]] requested during the review process for FFmpeg inclusion were never made.

[[Category:FFmpeg]]

FFmpeg filter HOWTO

2007-08-20T16:49:10Z

Koorogi: Explain the workings of a simple filter

This page is meant as an introduction of writing filters for [[libavfilter]]. This is a work in progress, but should at least point you in the right direction for writing simple filters.

== Definition of a filter ==

=== AVFilter ===
All filters are described by an AVFilter structure. This structure gives information needed to initialize the filter, and information on the entry points into the filter code. This structure is declared in libavfilter/avfilter.h:

typedef struct
{
char *name; ///< filter name
char *author; ///< filter author

int priv_size; ///< size of private data to allocate for the filter

int (*init)(AVFilterContext *ctx, const char *args, void *opaque);
void (*uninit)(AVFilterContext *ctx);

const AVFilterPad *inputs; ///< NULL terminated list of inputs. NULL if none
const AVFilterPad *outputs; ///< NULL terminated list of outputs. NULL if none
} AVFilter;

=== AVFilterPad ===
Let's take a quick look at the AVFilterPad structure, which is used to describe the inputs and outputs of the filter. This is also defined in libavfilter/avfilter.h:

typedef struct AVFilterPad
{
char *name;
int type;

int min_perms;
int rej_perms;

int *(*query_formats)(AVFilterLink *link);

void (*start_frame)(AVFilterLink *link, AVFilterPicRef *picref);
AVFilterPicRef *(*get_video_buffer)(AVFilterLink *link, int perms);
void (*end_frame)(AVFilterLink *link);
void (*draw_slice)(AVFilterLink *link, int y, int height);

int (*request_frame)(AVFilterLink *link);

int (*config_props)(AVFilterLink *link);
} AVFilterPad;

The actual definition in the header file has doxygen comments describing each entry point, its purpose, and what type of pads it is relevant for. These fields are relevant for all pads:

{| border="1"
!Field
!Description
|-
|name
|Name of the pad. No two inputs should have the same name, and no two outputs should have the same name.
|-
|type
|Only AV_PAD_VIDEO currently.
|-
|query_formats
|Returns a list of colorspaces supported on the pad.
|-
|config_props
|Handles configuration of the link connected to the pad
|}

Fields only relevant to input pads are:

{| border="1"
!Field
!Description
|-
|min_perms
|Minimum permissions required to a picture received as input.
|-
|rej_perms
|Permissions not accepted on pictures received as input.
|-
|start_frame
|Called when a frame is about to be given as input.
|-
|draw_slice
|Called when a slice of frame data has been given as input.
|-
|end_frame
|Called when the input frame has been completely sent.
|-
|get_video_buffer
|Called by the previous filter to request memory for a picture.
|}

Fields only relevant to output pads are:

{| border="1"
!Field
!Description
|-
|request_frame
|Requests that the filter output a frame.
|}

== Picture buffers ==

=== Reference counting ===
All pictures in the filter system are reference counted. This means that there is a picture buffer with memory allocated for the image data, and various filters can own a reference to the buffer. When a reference is no longer needed, its owner frees the reference. When the last reference to a picture buffer is freed, the filter system automatically frees the picture buffer.

=== Permissions ===
The upshot of multiple filters having references to a single picture is that they will all want some level of access to the image data. It should be obvious that if one filter expects to be able to read the image data without it changing that no other filter should write to the image data. The permissions system handles this.

In most cases, when a filter prepares to output a frame, it will request a buffer from the filter to which it will be outputting. It specifies the minimum permissions it needs to the buffer, though it may be given a buffer with more permissions than the minimum it requested.

When it wants to pass this buffer to another filter as output, it creates a new reference to the picture, possibly with a reduced set of permissions. This new reference will be owned by the filter receiving it.

So, for example, for a filter which drops frames if they are similar to the last frame it output, it would want to keeps its own reference to a picture after outputting it, and make sure that no other filter modified the buffer either. It would do this by requesting the permissions AV_PERM_READ|AV_PERM_WRITE|AV_PERM_PRESERVE for itself, and removing the AV_PERM_WRITE permission from any references it gave to other filters.

The available permissions are:
{| border="1"
!Permission
!Description
|-
|AV_PERM_READ
|Can read the image data.
|-
|AV_PERM_WRITE
|Can write to the image data.
|-
|AV_PERM_PRESERVE
|Can assume that the image data will not be modified by other filters. This means that no other filters should have the AV_PERM_WRITE permission.
|-
|AV_PERM_REUSE
|The filter may output the same buffer multiple times, but the image data may not be changed for the different outputs.
|-
|AV_PERM_REUSE2
|The filter may output the same buffer multiple times, and may modify the image data between outputs.
|}

== Filter Links ==
A filter's inputs and outputs are connected to those of another filter through the AVFilterLink structure:

typedef struct AVFilterLink
{
AVFilterContext *src; ///< source filter
unsigned int srcpad; ///< index of the output pad on the source filter

AVFilterContext *dst; ///< dest filter
unsigned int dstpad; ///< index of the input pad on the dest filter

int w; ///< agreed upon image width
int h; ///< agreed upon image height
enum PixelFormat format; ///< agreed upon image colorspace

AVFilterPicRef *srcpic;

AVFilterPicRef *cur_pic;
AVFilterPicRef *outpic;
};

The src and dst members indicate the filters at the source and destination ends of the link, respectively. The srcpad indicates the index of the output pad on the source filter to which the link is connected. Likewise, the dstpad indicates the index of the input pad on the destination filter.

When two filters are connected, they need to agree upon the dimensions of the image data they'll be working with, and the format that data is in. Once this has been agreed upon, these parameters are stored in the link structure.

The srcpic member is used internally by the filter system, and should not be accessed directly.

The cur_pic member is for the use of the destination filter. When a frame is currently being sent over the link (ie. starting from the call to start_frame() and ending with the call to end_frame()), this contains the reference to the frame which is owned by the destination filter.

The outpic member is described in the following tutorial on writing a simple filter.

== Writing a simple filter ==

=== Default filter entry points ===
Because the majority of filters that will probably be written will take exactly one input, and produce exactly one output, and output one frame for every frame received as input, the filter system provides a number default entry points to ease the development of such filters.

{| border="1"
!Entry point
!Actions taken by the default implementation
|-
|request_frame()
|Request a frame from the previous filter in the chain.
|-
|query_formats() on output pad
|Return a list of formats indicating that the format currently used on the input pad is the only supported output format.
|-
|start_frame()
|Request a buffer to store the output frame in. A reference to this buffer is stored in the outpic member of the link hooked to the filter's output. The next filter's start_frame() callback is called and given a reference to this buffer.
|-
|end_frame()
|Calls the next filter's end_frame() callback. Frees the reference to the outpic member of the output link, if it was set by (ie. if the default start_frame() is used). Frees the cur_pic reference in the input link.
|-
|get_video_buffer()
|Returns a buffer with the AV_PERM_READ permission in addition to all the requested permissions.
|-
|config_props() on output pad
|Sets the image dimensions for the output link to the same as on the filter's input.
|}

=== The vf_negate filter ===
Having looked at the data structures and callback functions involved, let's take a look at an actual filter. The vf_negate filter inverts the colors in a video. It has one input, and one output, and outputs exactly one frame for every input frame. In this way, it's fairly typical, and can takea dvantage of many of the default callback implementations offered by the filter system.

First, let's take a look at the AVFilter structure at the bottom of the libavfilter/vf_negate.c file:

AVFilter avfilter_vf_negate =
{
.name = "negate",
.author = "Bobby Bingham",

.priv_size = sizeof(NegContext),

.inputs = (AVFilterPad[]) {{ .name = "default",
.type = AV_PAD_VIDEO,
.draw_slice = draw_slice,
.query_formats = query_formats,
.config_props = config_props,
.min_perms = AV_PERM_READ, },
{ .name = NULL}},
.outputs = (AVFilterPad[]) {{ .name = "default",
.type = AV_PAD_VIDEO, },
{ .name = NULL}},
};

Here, you can see that the filter is named "negate," and it needs sizeof(NegContext) bytes of data to store its context. In the list of inputs and outputs, a pad whose name is set to NULL indicates the end of the list, so this filter has exactly one input and one output. If you look closely at the pad definitions, you will see that fairly few callback functions are actually specified. Because of the simplicity of the filter, the defaults can do most of the work for us.

Let us take a look at the callback function it does define.

==== query_formats() on an input pad ====
static int *query_formats(AVFilterLink *link)
{
return avfilter_make_format_list(10,
PIX_FMT_YUV444P, PIX_FMT_YUV422P, PIX_FMT_YUV420P,
PIX_FMT_YUV411P, PIX_FMT_YUV410P,
PIX_FMT_YUVJ444P, PIX_FMT_YUVJ422P, PIX_FMT_YUVJ420P,
PIX_FMT_YUV440P, PIX_FMT_YUVJ440P);
}

This simply calls avfilter_make\format_list(). This function takes as its first parameter the number of formats which will follow as the remaining parameters. The return value is a list of formats, terminated with -1, which is suitable for returning from query_formats(). As you can see, this filter supports a number of planar YUV colorspaces, including JPEG YUV colorspaces (the ones with a 'J' in the names).

Notice that the filter definition did not define a query_formats() for the output pad. In this case, the default will report that the output pad must operate in the same colorspace as the input.

==== config_props() on an input pad ====
The config_props() on an input pad is responsible for verifying that the properties of the input pad are supported by the filter, and to make any updates to the filter's context which are necessary for the link's properties.

TODO: quick explanation of YUV colorspaces, chroma subsampling, difference in range of YUV and JPEG YUV.

Let's take a look at the way in which this filter stores its context:

typedef struct
{
int offY, offUV;
int hsub, vsub;
} NegContext;

That's right. The priv_size member of the AVFilter structure tells the filter system how many bytes to reserve for this structure. The hsub and vsub members are used for chroma subsampling, and the offY and offUV members are used for handling the difference in range between YUV and JPEG YUV. Let's see how these are set in the input pad's config_props:

static int config_props(AVFilterLink *link)
{
NegContext *neg = link->dst->priv;

avcodec_get_chroma_sub_sample(link->format, &neg->hsub, &neg->vsub);

switch(link->format) {
case PIX_FMT_YUVJ444P:
case PIX_FMT_YUVJ422P:
case PIX_FMT_YUVJ420P:
case PIX_FMT_YUVJ440P:
neg->offY =
neg->offUV = 0;
break;
default:
neg->offY = -4;
neg->offUV = 1;
}

return 0;
}

This simply calls avcodec_get_chroma_sub_sample() to get the chroma subsampling shift factors, and stores those in the context. It then stores a set of offsets for compensating for different luma/chroma value ranges for JPEG YUV, and a different set of offsets for other YUV colorspaces. It returns zero to indicate success, because there are no possible input cases which this filter cannot handle.

==== draw_slice() ====
Finally, the function which actually does the processing for the filter, draw_slice():

static void draw_slice(AVFilterLink *link, int y, int h)
{
NegContext *neg = link->dst->priv;
AVFilterPicRef *in = link->cur_pic;
AVFilterPicRef *out = link->dst->outputs[0]->outpic;
uint8_t *inrow, *outrow;
int i, j, plane;

/* luma plane */
inrow = in-> data[0] + y * in-> linesize[0];
outrow = out->data[0] + y * out->linesize[0];
for(i = 0; i < h; i ++) {
for(j = 0; j < link->w; j ++)
outrow[j] = 255 - inrow[j] + neg->offY;
inrow += in-> linesize[0];
outrow += out->linesize[0];
}

/* chroma planes */
for(plane = 1; plane < 3; plane ++) {
inrow = in-> data[plane] + (y >> neg->vsub) * in-> linesize[plane];
outrow = out->data[plane] + (y >> neg->vsub) * out->linesize[plane];

for(i = 0; i < h >> neg->vsub; i ++) {
for(j = 0; j < link->w >> neg->hsub; j ++)
outrow[j] = 255 - inrow[j] + neg->offUV;
inrow += in-> linesize[plane];
outrow += out->linesize[plane];
}
}

avfilter_draw_slice(link->dst->outputs[0], y, h);
}

The y parameter indicates the top of the current slice, and the h parameter the slice's height. Areas of the image outside this slice should not be assumed to be meaningful (though a method to allow this assumption in order to simplify boundary cases for some filters is coming in the future).

This sets inrow to point to the beginning of the first row of the slice in the input, and outrow similarly for the output. Then, for each row, it loops through all the pixels, subtracting them from 255, and adding the offset which was determined in config_props() to account for different value ranges.

It then does the same thing for the chroma planes. Note how the width and height are shifted right to account for the chroma subsampling.

Once the drawing is completed, the slice is sent to the next filter by calling avfilter_draw_slice().

FFmpeg filter HOWTO

2007-08-19T20:35:18Z

Koorogi: add information on filter links

FFmpeg filter HOWTO

2007-08-19T20:04:43Z

Koorogi: Begin documenting how to write a filter for libavfilter

TX2

2007-04-20T09:53:00Z

Koorogi: document the TX2 image format

===Uses===
This format is used in Disgaea and other games by Nippon Ichi Software.

===Format===
The file consists of:
# a file header
# one or more color palettes as indicated in the header
# image data

The file may have garbage data after the end of the image. This should be discarded.

====Header====
Multibyte values in this format are stored with little-endian byte ordering.

typedef struct {
uint16_t width;
uint16_t height;
uint16_t colors; /* only values 16 and 256 have been observed */
uint16_t unknown;
uint16_t colors2; /* purpose unknown, but always seems to be the same as colors */
uint16_t palettes; /* number of palettes. only 1 and 16 observed thus far */
uint32_t padding; /* always zero */
} HEADER;

====Palette====
The palettes follow immediately after the header, and all have the number of entries indicated by the color field in the file header. The palettes have no header of their own, and instead simply consist of each color:

uint32_t color; /* 0xAABBGGRR in little endian */

In many images with no apparent need for transparency, the alpha value is always 0x80. Images which truly make use of the alpha channel have been observed, but not explored in detail yet.

====Image Data====
For 256 color images, each byte is a palette index, with bits 4 and 5 swapped. For the purposes of implementing a parser for this format, it is probably more efficient to reorder the palette while reading it, rather than swapping bits for each pixel.

For 16 color images, each byte gives two palette indexes - the first in the low nibble, the second in the high nibble.

[[Category:Image Formats]]
[[Category:Game Formats]]

Talk:FFmpeg Summer Of Code 2007

2007-03-15T01:58:06Z

Koorogi: /* Dirac */

== TIFF/TGA encoder? ==

Why? It seems too small a task to be done in the course of SoC. --[[User:Kostya|Kostya]] 09:20, 7 March 2007 (EST)

: True, I'll rework it. --[[User:Merbanan|Merbanan]] 11:44, 7 March 2007 (EST)

== DPX Encoder/Decoder ==
From what I understand this is the output of most film scanners. Having not used one I'm not 100% sure of this, and maybe someone could clarify, but my understanding is that film scanners output a collection of these files (I'm unsure if they're put in a container or not). There are already open source implementations in imagemagick and graphics magick.

: added some info and links - [[User:Compn|Compn]] 20:25, 8 March 2007 (EST)

== ATRAC3 ==
Should http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/44526 be added to small tasks?
[[User:Ce|Ce]] 04:41, 8 March 2007 (EST)

: No, I'll fix it eventually, it's not that simple.--[[User:Merbanan|Merbanan]] 06:46, 8 March 2007 (EST)

== Qualification Tasks ==
is reviewing and fixing up old unapplied ffmpeg patches not a good small task? [[User:Compn|Compn]] 20:25, 8 March 2007 (EST)

:I wanted to ask the same question a minute ago;-) The QTRLE patch is not ready yet, and there is no patch for an DNxHD ENcoder in the mailing list. I originally wrote that someone with better knowledge should of course remove those proposals if he thinks they are bad, but I still think they probably tell something about applicants (and that Baptiste possibly didn't read Merbanans "rules"). --[[User:Ce|Ce]] 20:31, 8 March 2007 (EST)

== Photo Codecs ==

I see several GSoC project proposals for encoding and decoding a variety of images. Is this truly appropriate for the FFmpeg project, which traditionally focuses on sequences of moving pictures vs. single images? I know that FFmpeg can put a movie together from a sequence of still pictures, or dump a movie into a series of still pictures. Are we hoping to do the same with HDR images? Otherwise, this type of work seems best left to dedicated photo processing projects. --[[User:Multimedia Mike|Multimedia Mike]] 13:36, 9 March 2007 (EST)

: And they are designed to operate with colour formats not currently supported by FFmpeg (like 16 bit per component). But JPEG-2000 and HD-Photo are likely to be used in movies so their support is undoubtedly useful. BTW, how do you plan to use qualification projects? --[[User:Kostya|Kostya]] 14:09, 9 March 2007 (EST)

::I'm supportive of something like JPEG-2000 (which uses a colorspace FFmpeg does not presently support, IIRC) since I know that there are plans to include that with certain types of movies. I'm not as enthusiastic about graphic formats that are not known to be encoded as sequences of images in a video file. As for the qualifier projects, we are hoping to weed out unqualified applicants by asking that they perform a task from the list. --[[User:Multimedia Mike|Multimedia Mike]] 16:02, 9 March 2007 (EST)

: i was thinking of that exact feature (convert dpx/tif/exr to h264). also i wonder if the encoder feature could be used for filmmaking using ffmpeg ? e.g. grab from camera right to HD image format? or is this too high end/specialized/low user count? --[[User:Compn|Compn]] 18:52, 9 March 2007 (EST)

:: You know me-- low user count is not a legit reason for discounting a feature. :-) If there is a legitimate video-type app for a certain feature I think that makes it more relevant to FFmpeg. --[[User:Multimedia Mike|Multimedia Mike]] 19:39, 9 March 2007 (EST)

::: btw i dont think they need to be SOC projects. exr and dpx are open and have gpl libs (dunno about microsoft hd photo). maybe just a qualifying task or just a wishlist. --[[User:Compn|Compn]] 22:53, 9 March 2007 (EST)

== AAC ==

I think finishing AAC support would be a good SoC project. The decoder started in 2006 is far from finished and the project appears to be quite complex. Baptiste appears to disagree, can we come to a consensus here? Does anybody know the exact status of the AAC implementation from 2006? -- [[User:DonDiego|DonDiego]] 10:11, 12 March 2007 (EDT)

: I think the LC part is almost complete. Adding He-AAC(+) features could be a SoC task.--[[User:Merbanan|Merbanan]] 17:05, 12 March 2007 (EDT)

==Dirac==
is dirac even finished spec wise? i'd rather see a decoder for the files in the wild (eac3, aac, gsm, 263, indeo)...

: Their site says it's "essentially complete". Granted, it's not out there in the wild yet, but I think that's mostly a matter of time, so why not get a decoder in now? It's not like libavcodec doesn't already have decoders for a number of other rather obscure formats already. -- [[User:Koorogi|Koorogi]] 21:58, 14 March 2007 (EDT)