MultimediaWiki - User contributions [en]

FFmpeg Summer of Code 2014

2014-02-05T19:04:25Z

Reimar: /* Hardware Acceleration API Software/Tracing Implementation */

This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.

= Introduction =

FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.

[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.

This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the GSoC Timeline for important dates.

== Information for Students ==

=== Getting Started ===

0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.

1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.

2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.

3. '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.

=== Qualification Tasks ===

In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.

The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.

=== Contacting FFmpeg ===

If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:

* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]
* '''IRC:''' ''#ffmpeg-devel'' on Freenode
* '''FFmpeg GSoC Admins:''' TBA

You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.

= Mentored Projects =

This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.

== H.264 Multiview Video Coding (MVC) ==

<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div>

'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].

'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.

'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)

== Animated Portable Network Graphics (APNG) ==

<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div>

'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.

'''Specification:''' https://wiki.mozilla.org/APNG_Specification

'''Expected results:'''
* APNG demuxer
** implement robust probing:
*** PNG images are not misdetected as APNG animations
*** APNG animations are not misdetected as PNG images
** splits stream into sensible packets (so they can be easily reused in APNG muxer)
** survives fuzzing (zzuf)
** add FATE coverage, coverage should be at least 70%
** test code under valgrind so no invalid reads/writes happen

* APNG decoder
** use existing PNG decoder code (write decoder in same file)
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)
** error handling
** survives fuzzing (zzuf)
** add test for FATE, coverage should be at least 75%
** CRC checksum validation
** test code under valgrind so no invalid reads/writes happen

* APNG muxer && APNG encoder
** use existing PNG encoder code (write encoder in same file)
** write compliant files, make sure they play correctly in major web browsers that support APNG
** add test for FATE

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.

'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)

== FFv1 P frame support ==

'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.

'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.

'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.

'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.

'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' TBA

== Misc Libavfilter extension ==

<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div>

'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.

Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.

Some ideas for more filters:
* a frequency filtering domain filter relying on the FFT utils in libavcodec
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI
* a lua scripting filter, which allows to implement filtering custom logic in lua

For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].

'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.

'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

'''Qualification Task:''' write or port one or more filters

'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)

== Subtitles ==

'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.

'''Expected results:'''
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.
* Update the API to get rid of the clumsy internal text representation of styles
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext

'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.

'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.

'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)

== Postproc optimizations ==

<div class="floatright">[[Image:PostProc.jpg]]</div>

'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).

'''Expected results:'''
* Convert all gcc inline asm in libpostproc to YASM.
* Restructure the code so that it works with block sizes compatible with modern SIMD.
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.

'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.

'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.

'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

<br clear="all">

== Bayer RGB colorspaces ==

<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div>

'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.

'''Expected results:'''
* Rebase existing patches
* Implement high quality bayer transformations in libswscale (plain C)
* Add bayer formats to the libavutil pixfmt enumeration routines
* SIMD optimizations of the libswscale transformations
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)

Optional goodies:
* Extend TIFF decoder to support DNG-Bayer format
* Support a popular proprietary camera format (many to choose from; see dcraw project)

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale

'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

<br clear="all">

== MPEG-4 ALS encoder ==

'''Description:'''
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.

'''Expected results:'''

* Update the existing encoder to fit into the current codebase.
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.
* Ensure the FFmpeg decoder processes all generated files without warnings.
* Enhance the rudimentary feature set of the encoder.

'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.

'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder

'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

<br clear="all">

== Hardware Acceleration API Software/Tracing Implementation ==

'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.

The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.

A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.

'''Prerequisites:''' C coding skills, basic familiarity with git

'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable

'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)

== Hardware Acceleration (hwaccel) API v2 ==

<div class="floatright">[[Image:Hardware.jpg]]</div>

'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.

The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.

'''Expected results:'''
* FFmpeg core library (libavcodec):
** Core API extensions and improvements
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications
*** Add new tokens describing hardware accelerators
*** Add new flags exposing HW capabilities like download/upload
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers
** Port hwaccels to v2 infrastructure
*** Port VA-API decoders to v2 infrastructure
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1
*** Provide download capability through ''vaGetImage()''
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications

* FFmpeg applications:
** Integrate hardware acceleration into ffplay
*** Create a video-output (VO) infrastructure to ffplay
*** Port the SDL renderer to the new VO infrastructure
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''
** Integrate hardware acceleration into ffmpeg
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use

'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.

'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.

'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA

<br clear="all">

== Hardware Accelerated Video Encoding with VA-API ==

'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.

'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.
* MPEG-2 encoding:
** Add basic encoding with I/P frames (handle the ''-g'' option)
** Add support for B frames (handle the ''-bf'' option)
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)
** (Optionally) add support for interlaced contents
* H.264 encoding:
** Add basic encoding with I/P frames (handle the ''-g'' option)
** Add support for B frames (handle the ''-bf'' option)
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities
** Work on HRD conformance. May require to write an independent tool to assess that
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.
* FFmpeg applications:
** Define common hwaccel interface for encoding
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application

'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.

'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.

'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly Tushar Gohad

<br clear="all"/>

== AAC Improvements ==

'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples
* AAC SSR decoder
* AAC 960/120 MDCT window

'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.

'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.

'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

<br clear="all"/>

== DTS / DCA Improvements ==

'''Description:''' FFmpeg contains a DTS decoder.
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]
** Add support for X96 extension (96khz)
** Add support for XLL extension (lossless)
** Add support for pure DTS-HD streams that do not contain a DTS core
** Add support for multiple assets
** Add support for LBR extension

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

== MXF Demuxer Improvements ==

'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.

The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.

'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.

'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA

<br clear="all"/>

= Unmentored Projects =

This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].

== glplay ==

<div class="floatleft">[[Image:Opengl_logo.jpg]]</div>

'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.

'''Mentor:''' TBD Backup: Reimar Döffinger

<br clear="all">

== TrueHD encoder ==

'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.

== Opus decoder ==

<div class="floatright">[[Image:Opus.png]]</div>

'''Description:''' Opus decoding is currently supported through the external libopus library
* Write a native decoder, continue working on the existing unfinished implementation
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.

== VC-1 interlaced ==

'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.
* Implement missing interlace features
* Make more reference samples bit-exact
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.

== JPEG 2000 ==

<div class="floatleft">[[Image:Jpeg2000.jpg]]</div>

'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.

<br clear="all">

== VP8L ==

'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.

== Your Own Project Idea ==

A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.

Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.

[[Category:FFmpeg]]

FFmpeg Summer of Code 2014

2014-02-05T18:32:51Z

Reimar: Add Hardware Acceleration API Software/Tracing Implementation

This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.

= Introduction =

FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.

[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.

This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the GSoC Timeline for important dates.

== Information for Students ==

=== Getting Started ===

0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.

1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.

2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.

3. '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.

=== Qualification Tasks ===

In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.

The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.

=== Contacting FFmpeg ===

If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:

* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]
* '''IRC:''' ''#ffmpeg-devel'' on Freenode
* '''FFmpeg GSoC Admins:''' TBA

You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.

= Mentored Projects =

This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.

== H.264 Multiview Video Coding (MVC) ==

<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div>

'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].

'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.

'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)

== Animated Portable Network Graphics (APNG) ==

<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div>

'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.

'''Specification:''' https://wiki.mozilla.org/APNG_Specification

'''Expected results:'''
* APNG demuxer
** implement robust probing:
*** PNG images are not misdetected as APNG animations
*** APNG animations are not misdetected as PNG images
** splits stream into sensible packets (so they can be easily reused in APNG muxer)
** survives fuzzing (zzuf)
** add FATE coverage, coverage should be at least 70%
** test code under valgrind so no invalid reads/writes happen

* APNG decoder
** use existing PNG decoder code (write decoder in same file)
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)
** error handling
** survives fuzzing (zzuf)
** add test for FATE, coverage should be at least 75%
** CRC checksum validation
** test code under valgrind so no invalid reads/writes happen

* APNG muxer && APNG encoder
** use existing PNG encoder code (write encoder in same file)
** write compliant files, make sure they play correctly in major web browsers that support APNG
** add test for FATE

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.

'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)

== FFv1 P frame support ==

'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.

'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.

'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.

'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.

'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' TBA

== Misc Libavfilter extension ==

<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div>

'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.

Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.

Some ideas for more filters:
* a frequency filtering domain filter relying on the FFT utils in libavcodec
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI
* a lua scripting filter, which allows to implement filtering custom logic in lua

For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].

'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.

'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

'''Qualification Task:''' write or port one or more filters

'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)

== Subtitles ==

'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.

'''Expected results:'''
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.
* Update the API to get rid of the clumsy internal text representation of styles
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext

'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.

'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.

'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)

== Postproc optimizations ==

<div class="floatright">[[Image:PostProc.jpg]]</div>

'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).

'''Expected results:'''
* Convert all gcc inline asm in libpostproc to YASM.
* Restructure the code so that it works with block sizes compatible with modern SIMD.
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.

'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.

'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.

'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

<br clear="all">

== Bayer RGB colorspaces ==

<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div>

'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.

'''Expected results:'''
* Rebase existing patches
* Implement high quality bayer transformations in libswscale (plain C)
* Add bayer formats to the libavutil pixfmt enumeration routines
* SIMD optimizations of the libswscale transformations
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)

Optional goodies:
* Extend TIFF decoder to support DNG-Bayer format
* Support a popular proprietary camera format (many to choose from; see dcraw project)

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale

'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)

<br clear="all">

== MPEG-4 ALS encoder ==

'''Description:'''
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.

'''Expected results:'''

* Update the existing encoder to fit into the current codebase.
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.
* Ensure the FFmpeg decoder processes all generated files without warnings.
* Enhance the rudimentary feature set of the encoder.

'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.

'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder

'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

<br clear="all">

== Hardware Acceleration API Software/Tracing Implementation ==

'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.

The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.

A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.

'''Prerequisites:''' C coding skills, basic familiarity with git

'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable

'''Mentor:''' Reimar Döffinger

== Hardware Acceleration (hwaccel) API v2 ==

<div class="floatright">[[Image:Hardware.jpg]]</div>

'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.

The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.

'''Expected results:'''
* FFmpeg core library (libavcodec):
** Core API extensions and improvements
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications
*** Add new tokens describing hardware accelerators
*** Add new flags exposing HW capabilities like download/upload
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers
** Port hwaccels to v2 infrastructure
*** Port VA-API decoders to v2 infrastructure
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1
*** Provide download capability through ''vaGetImage()''
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications

* FFmpeg applications:
** Integrate hardware acceleration into ffplay
*** Create a video-output (VO) infrastructure to ffplay
*** Port the SDL renderer to the new VO infrastructure
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''
** Integrate hardware acceleration into ffmpeg
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use

'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.

'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.

'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA

<br clear="all">

== Hardware Accelerated Video Encoding with VA-API ==

'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.

'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.
* MPEG-2 encoding:
** Add basic encoding with I/P frames (handle the ''-g'' option)
** Add support for B frames (handle the ''-bf'' option)
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)
** (Optionally) add support for interlaced contents
* H.264 encoding:
** Add basic encoding with I/P frames (handle the ''-g'' option)
** Add support for B frames (handle the ''-bf'' option)
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities
** Work on HRD conformance. May require to write an independent tool to assess that
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.
* FFmpeg applications:
** Define common hwaccel interface for encoding
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application

'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.

'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.

'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly Tushar Gohad

<br clear="all"/>

== AAC Improvements ==

'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples
* AAC SSR decoder
* AAC 960/120 MDCT window

'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.

'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.

'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

<br clear="all"/>

== DTS / DCA Improvements ==

'''Description:''' FFmpeg contains a DTS decoder.
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]
** Add support for X96 extension (96khz)
** Add support for XLL extension (lossless)
** Add support for pure DTS-HD streams that do not contain a DTS core
** Add support for multiple assets
** Add support for LBR extension

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)

== MXF Demuxer Improvements ==

'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.

The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.

'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.

'''Prerequisites:''' C coding skills, basic familiarity with git.

'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.

'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)

'''Backup Mentor:''' TBA

<br clear="all"/>

= Unmentored Projects =

This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].

== glplay ==

<div class="floatleft">[[Image:Opengl_logo.jpg]]</div>

'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.

'''Mentor:''' TBD Backup: Reimar Döffinger

<br clear="all">

== TrueHD encoder ==

'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.

== Opus decoder ==

<div class="floatright">[[Image:Opus.png]]</div>

'''Description:''' Opus decoding is currently supported through the external libopus library
* Write a native decoder, continue working on the existing unfinished implementation
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.

== VC-1 interlaced ==

'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.
* Implement missing interlace features
* Make more reference samples bit-exact
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.

== JPEG 2000 ==

<div class="floatleft">[[Image:Jpeg2000.jpg]]</div>

'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.

<br clear="all">

== VP8L ==

'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.

== Your Own Project Idea ==

A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.

Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.

[[Category:FFmpeg]]

FFmtech TODO

2013-09-15T22:37:13Z

Reimar:

A list of (smallish?) tasks to do for the foundation, to pick whenever someone has time.

* review web page
* upload the bylaws properly to the web page
* add slightly more detailed descriptions of the board members
* link to this page from some appropriate places?
* plan next board meeting, discuss topics on list
* add more tasks to this list

FFmtech TODO

2013-09-15T22:33:17Z

Reimar:

A list of (smallish?) tasks to do for the foundation, to pick whenever someone has time.

* review web page
* upload the bylaws properly to the web page
* add slightly more detailed descriptions of the board members
* plan next board meeting, discuss topics on list
* add more tasks to this list

FFmtech TODO

2013-09-15T22:32:43Z

Reimar:

USM

2013-09-03T19:18:55Z

Reimar:

USM Container format

This format is used by e.g. "The Witcher 2".

It contains ordinary MPEG (probably MPEG-2) video, with the encoder string "TMPGEXE" being very obvious.

The file consists of (in the header nested) chunks.

Each main chunk is at least 64 byte aligned (though the design of the format should allow anything between including 0 and 64kB alignment if desired).

The format of a chunk is
* 4 byte identifier string
* 4 byte length of data not including 8 bytes for identifier and length, big-endian
* 2 byte unknown (always 0 for @UTF, 0x18 otherwise?)
* 2 byte padding bytes
* 4 byte probably type 1 = header, 2 = metadata, 3 = index, 0 = data
* 4 byte timestamp
* 4 byte unknown
* 8 byte unknown (always 0, padding?)

The known indentifier strings are:
* CRID Main header, usually 512 bytes long
* @UTF Used only as a sub-chunk, only 32 byte aligned. Header/meta data, consisting of first the (binary) real data, followed by a textual description of the fields.
* @SFV Video stream. If type is "data", the actual data seems to start only at offset 0x60.
* @SFA Audio stream. If type is "data", the actual data just like for metadata seems to start at offset 0x20.

An index (called SEEKINFO) seems to only exist for video (it is placed into a @SFV tag), probably assuming the audio to be correct interleaved.

[[Category:Container Formats]]

USM

2013-09-01T16:35:17Z

Reimar:

USM Container format

This format is used by e.g. "The Witcher 2".

It contains ordinary MPEG (probably MPEG-2) video, with the encoder string "TMPGEXE" being very obvious.

The file consists of (in the header nested) chunks.

Each main chunk is at least 64 byte aligned (though the design of the format should allow anything between including 0 and 64kB alignment if desired).

The format of a chunk is
* 4 byte identifier string
* 4 byte length of data not including 8 bytes for identifier and length, big-endian
* 2 byte unknown
* 2 byte padding bytes
* 4 byte probably type 1 = header, 2 = metadata, 3 = index, 0 = data
* 4 byte timestamp
* 4 byte unknown
* 8 byte unknown

The known indentifier strings are:
* CRID Main header, usually 512 bytes long
* @UTF Used only as a sub-chunk, only 32 byte aligned. Header/meta data, consisting of first the (binary) real data, followed by a textual description of the fields.
* @SFV Video stream. If type is "data", the actual data seems to start only at offset 0x60.
* @SFA Audio stream. If type is "data", the actual data just like for metadata seems to start at offset 0x20.

An index (called SEEKINFO) seems to only exist for video (it is placed into a @SFV tag), probably assuming the audio to be correct interleaved.

[[Category:Container Formats]]

USM

2013-09-01T16:33:17Z

Reimar:

USM Container format

This format is used by e.g. "The Witcher 2".

It contains ordinary MPEG (probably MPEG-2) video, with the encoder string "TMPGEXE" being very obvious.

The file consists of (in the header nested) chunks.

Each main chunk is at least 64 byte aligned (though the design of the format should allow anything between including 0 and 64kB alignment if desired).

The format of a chunk is
* 4 byte identifier string
* 4 byte length, big-endian
* 2 byte unknown
* 2 byte padding bytes
* 4 byte probably type 1 = header, 2 = metadata, 3 = index, 0 = data
* 4 byte timestamp
* 4 byte unknown
* 8 byte unknown

The known indentifier strings are:
* CRID main header
* @UTF Used only as a sub-chunk, only 32 byte aligned. Header/meta data, consisting of first the (binary) real data, followed by a textual description of the fields.
* @SFV Video stream. If type is "data", the actual data seems to start only at offset 0x60.
* @SFA Audio stream. If type is "data", the actual data just like for metadata seems to start at offset 0x20.

An index (called SEEKINFO) seems to only exist for video (it is placed into a @SFV tag), probably assuming the audio to be correct interleaved.

[[Category:Container Formats]]

USM

2013-09-01T16:32:08Z

Reimar:

USM Container format

This format is used by e.g. "The Witcher 2".
It contains ordinary MPEG (probably MPEG-2) video, with the encoder string "TMPGEXE" being very obvious.
The file consists of (in the header nested) chunks.
Each main chunk is at least 64 byte aligned (though the design of the format should allow anything between including 0 and 64kB alignment if desired).
The format of a chunk is
4 byte identifier string
4 byte length, big-endian
2 byte unknown
2 byte padding bytes
4 byte probably type 1 = header, 2 = metadata, 3 = index, 0 = data
4 byte timestamp
4 byte unknown
8 byte unknown

The known indentifier strings are:
CRID main header
@UTF Used only as a sub-chunk, only 32 byte aligned. Header/meta data, consisting of first the (binary) real data, followed by a textual description of the fields.
@SFV Video stream. If type is "data", the actual data seems to start only at offset 0x60.
@SFA Audio stream. If type is "data", the actual data just like for metadata seems to start at offset 0x20.

An index (called SEEKINFO) seems to only exist for video (it is placed into a @SFV tag), probably assuming the audio to be correct interleaved.

[[Category:Container Formats]]

USM

2013-09-01T16:29:57Z

Reimar:

USM Container format

This format is used by e.g. "The Witcher 2".
It contains ordinary MPEG (probably MPEG-2) video, with the encoder string "TMPGEXE" being very obvious.
The file consists of (in the header nested) chunks.
Each main chunk is at least 64 byte aligned (though the design of the format should allow anything between including 0 and 64kB alignment if desired).
The format of a chunk is
4 byte identifier string
4 byte length, big-endian
2 byte unknown
2 byte padding bytes
4 byte probably type 1 = header, 2 = metadata, 3 = index, 0 = data
4 byte timestamp
4 byte unknown
8 byte unknown

The known indentifier strings are:
CRID main header
@UTF Used only as a sub-chunk, only 32 byte aligned. Header/meta data, consisting of first the (binary) real data, followed by a textual description of the fields.
@SFV Video stream. If type is "data", the actual data seems to start only at offset 0x60.
@SFA Audio stream. If type is "data", the actual data just like for metadata seems to start at offset 0x20.

An index (calles SEEKINFO) seems to only exist for video (it is placed into a @SFV tag), probably assuming the audio to be correct interleaved.

FFmpeg Summer of Code 2013

2013-03-06T19:57:14Z

Reimar: /* glplay */

FFmpeg Summer of Code Ideas List

= Proposing Ideas =

If you have a project idea please contact a developer first at the #ffmpeg-devel IRC channel on Freenode or via the [http://ffmpeg.org/contact.html ffmpeg-devel mailing list]. A good source of ideas is the [https://ffmpeg.org/trac/ffmpeg/ FFmpeg bug tracker] and [[FFmpeg_Summer_of_Code_2012|FFmpeg Summer of Code 2012 Ideas List]].

When adding an idea follow this template for consistency:

== Example Title ==

'''Description:''' A few sentences or a short paragraph describing the task.

'''Expected results:''' Bulleted list or paragraph describing what the student is expected to achieve.

'''Prerequisites:''' Skills or knowledge required by student.

'''Mentor:''' List mentor and backup mentor if there is one and contact info such as IRC name or email address.

= GSoC task proposal ideas =

== Audio codecs ==

=== AAC ===

'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.
* AAC LD decoder
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples
* AAC SSR decoder
* AAC 960/120 MDCT window
* AAC multi-channel encoding
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.

=== DTS / DCA ===

'''Description:''' FFmpeg contains a DTS decoder and an experimental DTS encoder. Both are missing some features.
* DTS-HD decoder improvements: A possible qualification task is to implement ticket #1920
** Add support for X96 extension (96khz)
** Add support for XLL extension (lossless)
** Add support for pure DTS-HD streams that do not contain a DTS core
** Add support for multiple assets
** Add support for LBR extension
* DTS encoder improvements

=== MPEG-4 ALS encoder ===

'''Description:''' An ALS decoder was implemented several years ago, an encoder is still missing.

=== TrueHD encoder ===

'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.

=== Opus decoder ===

'''Description:''' Opus decoding is currently supported through the external libopus library
* Write a native decoder, continue working on the existing unfinished implementation
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.

== Video codecs ==

=== HEVC / H265 ===

'''Description:''' The specification was finished, FFmpeg needs a decoder for this new format.
* Write a basic decoder supporting I, P, and, only if time permits, B slices.
* It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.
As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.

=== H264 MVC ===

'''Description:''' MVC samples exist, the codec is used on Bluray media, FFmpeg is missing a decoder. Since this task also consists of some changes in the current architecture, it is especially important that this task is discussed on the ffmpeg-devel mailing list.
As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.

=== VC-1 interlaced ===

'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.
* Implement missing interlace features
* Make more reference samples bit-exact
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.

=== Animated Portable Network Graphics (APNG) ===

'''Description:''' FFmpeg currently does not support Animated PNGs.

'''Specification:''' https://wiki.mozilla.org/APNG_Specification

'''Expected results:'''
* APNG demuxer
** implement robust probing:
*** PNG images are not misdetected as APNG animations
*** APNG animations are not misdetected as PNG images
** splits stream into sensible packets (so they can be easily reused in APNG muxer)
** survives fuzzing (zzuf)
** add FATE coverage, coverage should be at least 70%
** test code under valgrind so no invalid reads/writes happen

* APNG decoder
** use existing PNG decoder code (write decoder in same file)
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)
** error handling
** survives fuzzing (zzuf)
** add test for FATE, coverage should be at least 75%
** CRC checksum validation
** test code under valgrind so no invalid reads/writes happen

* APNG muxer && APNG encoder
** use existing PNG encoder code (write encoder in same file)
** write compliant files, make sure they play correctly in major web browsers that support APNG
** add test for FATE

'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.

'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer

'''Mentor: [[User:Pbm|Paul B Mahol]]'''

=== JPEG 2000 ===

'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.

=== VP7 ===

'''Description:''' Not many VP7 samples are in the wild, but no open-source decoder exists although a specification exists. Write a decoder that reuses as much as possible of existing FFmpeg code, it is likely that functions of the existing decoders for On2-based formats will be useful.

== Other ==

=== glplay ===

'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.

'''Mentor:''' TBD Backup: Reimar Döffinger

=== Misc Libavfilter extension ===

'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.

Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.

Some ideas for more filters:
* a frequency filtering domain filter relying on the FFT utils in libavcodec
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI
* a lua scripting filter, which allows to implement filtering custom logic in lua

For more ideas check:
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].

'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.

'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

'''Qualification task:''' write or port one or more filter

'''Mentor:''' Stefano Sabatini (''saste'' on IRC). Backup: Clément Bœsch (''ubitux'' on IRC).

=== Subtitles ===

'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carry an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.

'''Expected results:'''
* Write the support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray.
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.
* Improve the API to facilitates a proper integration of subtitles into libavfilter.

'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.

'''Qualification task:''' write one subtitles demuxer or muxer, and do some improvements to the different layers of the subtitles (to make sure the whole chain is understood).

'''Mentor:''' Clément Bœsch (''ubitux'' on IRC).

[[Category:FFmpeg]]

Small FFmpeg Tasks

2012-05-08T18:39:56Z

Reimar: /* Programming Tasks */

This page contains ideas for small, relatively simple tasks for the [[FFmpeg]] project. People who might be interested in trying one of these tasks:
* Someone who wants to contribute to FFmpeg and needs to find a well-defined task to start with
* Someone who wishes to qualify for one of FFmpeg's coveted [[FFmpeg Summer Of Code|Summer of Code]] project slots
* An existing FFmpeg developer who has been away from the project for a while and needs a smaller task as motivation for re-learning the codebase

For other tasks of varying difficulty, see the [[Interesting Patches]] page.

'''If you would like to work on one of these tasks''', please take these steps:
* Subscribe to the [https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel FFmpeg development mailing list] and indicate your interest
* Ask [[User:Multimedia Mike|Multimedia Mike]] for a Wiki account so you can claim your task on this Wiki

'''If you would like to add to this list''', please be prepared to explain some useful details about the task. Excessively vague tasks with no supporting details will be ruthlessly deleted.

== Programming Tasks ==

=== Finish up a previous incomplete SoC project ===

Several SoC projects from previous years have not yet made it into FFmpeg. Taking any of them and finishing them up to the point that they can be included should make for a good qualification task. Check out the [[FFmpeg Summer Of Code]] overview page and look for the unfinished projects, like the TS muxer.

=== Add code to validate get_buffer usage of decoders ===
Change the default_get_buffer etc. functions to enforce the minimum guarantees the decoder requests.
E.g. if a decoder does not set FF_BUFFER_HINTS_READABLE, return a buffer without read permissions (using e.g. mprotect).
If the decoder does not use reget_buffer, always return a buffer initialized with random data.
If the decoder does not set FF_BUFFER_HINTS_PRESERVE, always destroy the buffer contents as soon as possible.
Make reget_buffer always fail if FF_BUFFER_HINTS_REUSABLE was not used.
Probably more things that could be done.

=== Generic Colorspace system ===
This task involves adding support more than 8 bits per component (Y on 10 bits, U on 10 bits, V on 10 bits for example)
and generic simple conversion to other colorspaces.

''Does this have to do with revising FFmpeg's infrastructure? If so, then it doesn't feel like a qualification task. If it's something simpler, then the vague description does not convey that simplicity. Please expound.'' --[[User:Multimedia Mike|Multimedia Mike]] 12:56, 25 February 2008 (EST)

''I don't think so, extending PixFmt to extended structure with finegrained description like depth, range values, colorspace, sample period, and write generic simple conversion from all formats to all others, like suggested by Michael on the mailing list. Conversion routine can be a good qualification task for video encoders/decoders. What do you think ?
--[[User:Bcoudurier|Baptiste Coudurier]] 00:30, 29 February 2008 (EST)

''* Adding the [[YCoCg]] colorspace (with different sized planes) for RGB sourced pictures would be nice too. [[User:Elte|Elte]] 07:15, 16 March 2009 (EDT)

=== Extend GIF Encoder and Decoder to support Animated GIFs ===

=== Implement a Vivo demuxer ===
Implement a demuxer for the [[Vivo]] file format. The best reference for understanding the format would be MPlayer's [http://svn.mplayerhq.hu/mplayer/trunk/libmpdemux/demux_viv.c?view=markup existing .viv demuxer].

This task corresponds to ticket 132: https://avcodec.org/trac/ffmpeg/ticket/132

''I am ready to help out with understanding MPlayer's demuxer, esp. MPlayer API stuff if necessary.
--[[User:Reimar|Reimar]] 15:46, 1 March 2008 (EST)

=== Port missing demuxers from MPlayer ===
MPlayer supports a few container formats in libmpdemux that are not yet present in libavformat. Porting them over and gettting them relicensed as LGPL or reimplementing them from scratch should make reasonable small tasks.

# TiVo --
# VIVO -- ''Daniel Verkamp has a patch for this''
# SL support for MPEG-TS (anyone got samples?)
# MNG -- ''Paul B Mahol is working on this''

=== Optimal Huffman tables for (M)JPEG ===
This task is outlined at http://guru.multimedia.cx/small-tasks-for-ffmpeg/ and is tracked in the issue tracker: http://roundup.libav.org/issue267

=== M95 Playback System ===
This task is to implement a playback subsystem for [[M95]] files. This will entail writing a new file demuxer and video decoder (the audio is already uncompressed), both of which should be fairly easy by FFmpeg standards. [[M95|The M95 page]] contains the specs necessary to complete this task and points to downloadable samples.

=== BRP Playback System ===
This task is to implement a playback subsystem for [[BRP]] files. This will entail writing a new file demuxer as well as a video decoder that can handle at least 2 variations of format data. Further, write an audio decoder for the custom DPCM format in the file. All of these tasks are considered fairly easy by FFmpeg standards. [[BRP|The BRP page]] contains the specs necessary to complete this task and points to downloadable samples for both known variations.

=== 16-bit VQA Video Decoder ===
Westwood [[VQA]] files are already supported. However, there are three variations of its custom video codec, only the first two of which are supported. This task involves implementing support for the third variation. Visit the VQA samples repository: http://samples.multimedia.cx/game-formats/vqa/ -- The files in the directories Tiberian Sun VQAs/, bladerunner/, and dune2000/ use the 3rd variation of this codec. The [[VQA|VQA page]] should link to all the details you need to support this format.

Discussion/patch:
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/89902/focus=90433

=== HNM4 Playback System ===
This task is to implement a playback subsystem for [[HNM4]] variant of the [[HNM]] format. This will entail writing a new file demuxer and video decoder, both of which are considered fairly easy by FFmpeg standards. [[HNM4|The HNM4 page]] contains the specs necessary to complete this task and links to downloadable samples.

=== Apple RPZA encoder ===
A patch was once sent to the ffmpeg-devel mailing list to include an encoder for the [[Apple RPZA]] video codec. That code can be found on the "[[Interesting Patches]]" page. This qualification task involves applying that patch so that it can compile with current HEAD and then cleaning it up per the standards of the project. Engage the mailing list to learn more about what to do.

=== QuickTime Edit List Support ===
Implement edit list support in the QuickTime demuxer (libavformat/mov.c). This involves parsing the 'elst' atom in a QuickTime file. For a demonstration of how this is a problem, download the file menace00.mov from http://samples.mplayerhq.hu/mov/editlist/ and play it with ffplay or transcode it with ffmpeg. Notice that the audio and video are ever so slightly out of sync. Proper edit list support will solve that. Other samples in that directory also presumably exhibit edit list-related bugs. The [http://xine.cvs.sourceforge.net/xine/xine-lib/src/demuxers/demux_qt.c?view=markup Xine demuxer] has support for this, it might be useful for hints.

(patch was submitted to ffmpeg-devel , around 14 March 2009)

=== Add wma fixed point decoder back into libavcodec ===
http://svn.rockbox.org/viewvc.cgi/trunk/apps/codecs/libwma/
Rockbox's fixed-point WMA decoder was adapted from the decoder in libavcodec.

=== VC1 timestamps in m2ts ===

Codec copy of VC1 from m2ts currently doesn't work. Either extend the VC1 parser to output/fix timestamps, or fix the timestamps from m2ts demuxing.

=== flip flag for upside-down codecs ===

<pre>about the flip, a patch that decodes images fliped when
codec_tag == ff_get_fourcc("GEOX") is welcome.
its a metter of 2lines manipulating data/linesize of imgages after
get_buffer() or something similar
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
</pre>
more info:
http://roundup.ffmpeg.org/roundup/ffmpeg/issue741

=== lavf-based concatenation tool ===

It would be nice to have some libavformat-based tool that would extract frames from multiple files (possible different containers as well) and put them into single one.

=== vcr1 encoder ===
According to this: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-February/063555.html vcr1 encoder is disabled, and won't compile if enabled. Michael would prefer to keep it around, and have someone grow it into full encoder.

=== implement some colorspace fourcc/codecs ===
some colorspace formats were uploaded to http://samples.mplayerhq.hu/V-codecs/
including:
CYUV.AVI is 8 Bit Interleaved 4:2:2
a12v.avi is 4:2:2:4 10 Bit Interleaved
auv2.avi is 4:2:2:4 8 Bit Interleaved
and V-codecs/yuv8/MAILTEST.AVI .

it might decode with current pixfmts, a sample commit is 9853bbb21a19d540850de60d3e9cf7c6ef9da7dc

a sample commit for adding new input formats to swscale is 4884b9e50d416f84e64bfaf546a03e490cb83a2f
the hunks 3 and 5 you do not need, they are optional special converters
also the change to isSupportedOut() you do not need
above will add a new input format

another example for adding an input format is a43fb6b37efa5b01f2c9bdc414570691229bcfab

=== Implement Phantom Cine demuxer and Bayer format support for swscale ===
The format is described here:
http://wiki.multimedia.cx/index.php?title=Phantom_Cine
It will need support for Bayer -> RGB conversion in swscale to make the demuxer useful though.

=== support for [[YCoCg]]/RGB colorspace in FFV1 ===
Add support for [[YCoCg]] and [[RGB]] encoded sources for the [[FFV1]] codec

This would add a free lossless intra-frame RGB codec for all supported platforms (most important OS X + Windows) which is often asked for video editing in video forums (e.g. slashcam.de)

=== [[IFF#ANIM|IFF ANIM]] decoder ===
Modify libavformat/iff.c to handle this chunk and write a decoder for the format. The wiki page at [[IFF#ANIM|IFF ANIM]] has links to more information and source code. Samples can be found at http://www-user.tu-chemnitz.de/~womar/projects/iffanim/iffanim_samplepack.zip .

=== port missing decoders/demuxers from other open source projects. ===

http://www.mega-nerd.com/libsndfile/#Features
Paris Audio File PAF
IRCAM SF
GNU Octave 2.0 MAT4
GNU Octave 2.1 MAT5
Portable Voice Format PVFSound
Designer II SD2
samples are here: http://www.mega-nerd.com/tmp/SoundFileCollection-20050711-0902.tgz

http://www.hawksoft.com/hawkvoice/
HVDI_VOICE_DATA- packet
[[GSM]]
LPC
CELP
LPC10

http://sourceforge.net/projects/vgmstream
150+ formats: http://vgmstream.svn.sourceforge.net/viewvc/vgmstream/readme.txt

http://www.imagemagick.org
http://www.graphicsmagick.org/formats.html
many image formats not supported yet.

http://gpac.sourceforge.net/
[[MPEG-4 BIFS]]
3GPP DIMS
[[LASeR]]
SAF
SVG
[[Synchronized Multimedia Integration Language|SMIL]]
VRML
X3D
XMT

http://adplug.sourceforge.net/
http://adplug.sourceforge.net/library/
many OPL2/OPL3 audio formats not supported yet.

http://mikmod.raphnet.net/
http://mikmod.raphnet.net/#features
many music pattern formats not supported yet.

http://www.fly.net/~ant/libs/audio.html#Game_Music_Emu
AY
GBS
GYM
HES
KSS
NSF, NSFE
SAP
[[SNES-SPC700 Sound Format]]
VGM, VGZ

=== libswscale PAL8 output ===

See the thread: "[RFC] libswscale palette output implementation":
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/101397

=== vloopback output support ===

vloopback is a linux kernel device which allows to create a virtual video device where
programs can write, and can be accessed as a normal video device:
http://www.lavrsen.dk/twiki/bin/view/Motion/VideoFourLinuxLoopbackDevice

This would allow to write the ffmpeg output to a vloopdevice and be displayed by some a
program reading from such device (e.g. skype, a voip client etc.).

An example of a program which uses vloopback:
http://www.ws4gl.org/

=== Port video filters from MPlayer/VLC/Mjpegtools/Effectv/etc etc to libavfilter ===

There are plenty programs providing their own filters, many of them may be easily ported to the
superior ;-) framework of libavfilter. Also may be possible to create wrappers around other libraries
(e.g. opencv, libgimp, libshowphoto, libaa).

=== rar/zip/gz/bz2 etc demuxer ===
there are still compressed files out there (zipped raw wav, multi-rar'ed videos etc). create a decompression demuxer for them.

=== Less common AAC decoder features ===

Add support to the AAC decoder for object type ER AAC LC or AAC LC 960.

=== arithmetic decoding (and encoding) for mjpeg ===
Following marker codes are not supported by our mjpeg decoder:
DAC, SOF9, SOF10, SOF11, SOF13, SOF14 and SOF15.

=== [[Enhanced Variable Rate Codec]] decoder ===
samples and specs available.

=== adobe http f4f segmented fragmentation dynamic streaming format ===
sample streams on http://www.fox.com . command line instructions for creating such files: http://help.adobe.com/en_US/HTTPStreaming/1.0/Using/WS9463dbe8dbe45c4c-c126f3b1260533756d-7ffc.html . spec is available under adobe NDA. not to be confused with freely available F4V specification. open source php to convert f4f to flv: https://github.com/svnpenn/dotfiles/blob/master/etc/AdobeHDS.php

=== get 3IV1 decoder working and benchmark ===
we have a decoder for 3IV1. its currently if 0'd in the mpeg4 decoder. your task is to test if it still builds and works, and fix it so that it does not slow down mpeg4 decoder if enabled. the end goal is to enable it by default.

== Reverse Engineering Tasks ==

=== Demuxer for csf format and video codec ===
This is partially analyzed in http://ffmpeg.org/trac/ffmpeg/ticket/1060

=== realplayer's ivr format and create a demuxer for it ===
samples on [[IVR]] page.

=== pick a random binary codec from mplayer ===
[[MPlayer]] has over 100 binary codecs which have no opensource decoder. pick one, find a sample and try to reverse engineer it. note that some work has been done on some codecs, and its a good idea to ask on the mailing list before starting.

=== emblaze demuxer/decoder from java code ===
samples and java decoder: http://samples.mplayerhq.hu/internets/emblaze/

== Non-Programming Tasks ==

=== Check Linux distributions for patches to ffmpeg ===

check various distros like Fedora, Ubuntu, Debian, Mint, Arch, Suse etc for patches to ffmpeg. write down location of patches so it can be checked on an annual basis. if patches are found , report to ffmpeg-devel mailing list or bug trac.

=== improve layout and accessability of ffmpeg website ===

test ffmpeg.org with various browsers, including screen readers and get it optimized and available to people with poor vision. check wording and general ease of use. for example putting large download links for users like vlc and firefox have.

=== restore and update compatability page on ffmpeg website ===

we used to have a page that detailed how to create files for other software players and operating systems. restore this page from git history and update it for new devices and standardized codecs (h264 is the preferred codec now).

=== contact large ffmpeg users for broken / unplayable samples ===
contact the largest users of ffmpeg, like youtube, facebook, archive.org, blip.tv and others and ask them for access to samples that do not decode correctly.

=== review sample request error messages ===
ffmpeg has an av_log_ask_for_sample generic log message to ask the user for a sample when there is a problem. your task is to review ffmpeg decoders and demuxers (and possibly other inputs) and replace regular av_log messages requesting samples with it. example commit here: http://ffmpeg.org/pipermail/ffmpeg-cvslog/2011-April/036509.html

=== write bluray and 3d howtos ===
write a document or wiki article or just link to some info on how to play + encode + rip bluray using ffmpeg/mplayer and the various bluray libs required. also a guide on how to use ffmpeg/mplayer/vlc to encode and play various 3D formats. including maybe some supported hardware screens/video cards w/ examples. includes updating this wiki page [[Blu Ray and HD-DVD Playback Status]]
[[How to make a 3d movie with ffmpeg]]

=== write fate breakage notification bot ===
script up a bot to scan http://fate.ffmpeg.org that tells #ffmpeg-devel when something breaks compilation in fate.

=== write ipad/iphone/ios howto ===
write up some documentation on how to compile ffmpeg/ffplay/ffserver for iOS. exact tool versions, command lines, library requirements. both native compilation on the device and cross-compile using OS X. an android howto for various devices would be useful too.

[[Category:FFmpeg]]
[[Category:Libav]]

Small FFmpeg Tasks

2012-04-29T12:36:43Z

Reimar: /* Reverse Engineering Tasks */

This page contains ideas for small, relatively simple tasks for the [[FFmpeg]] project. People who might be interested in trying one of these tasks:
* Someone who wants to contribute to FFmpeg and needs to find a well-defined task to start with
* Someone who wishes to qualify for one of FFmpeg's coveted [[FFmpeg Summer Of Code|Summer of Code]] project slots
* An existing FFmpeg developer who has been away from the project for a while and needs a smaller task as motivation for re-learning the codebase

For other tasks of varying difficulty, see the [[Interesting Patches]] page.

'''If you would like to work on one of these tasks''', please take these steps:
* Subscribe to the [https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel FFmpeg development mailing list] and indicate your interest
* Ask [[User:Multimedia Mike|Multimedia Mike]] for a Wiki account so you can claim your task on this Wiki

'''If you would like to add to this list''', please be prepared to explain some useful details about the task. Excessively vague tasks with no supporting details will be ruthlessly deleted.

== Programming Tasks ==

=== Finish up a previous incomplete SoC project ===

Several SoC projects from previous years have not yet made it into FFmpeg. Taking any of them and finishing them up to the point that they can be included should make for a good qualification task. Check out the [[FFmpeg Summer Of Code]] overview page and look for the unfinished projects, like the TS muxer.

=== Generic Colorspace system ===
This task involves adding support more than 8 bits per component (Y on 10 bits, U on 10 bits, V on 10 bits for example)
and generic simple conversion to other colorspaces.

''Does this have to do with revising FFmpeg's infrastructure? If so, then it doesn't feel like a qualification task. If it's something simpler, then the vague description does not convey that simplicity. Please expound.'' --[[User:Multimedia Mike|Multimedia Mike]] 12:56, 25 February 2008 (EST)

''I don't think so, extending PixFmt to extended structure with finegrained description like depth, range values, colorspace, sample period, and write generic simple conversion from all formats to all others, like suggested by Michael on the mailing list. Conversion routine can be a good qualification task for video encoders/decoders. What do you think ?
--[[User:Bcoudurier|Baptiste Coudurier]] 00:30, 29 February 2008 (EST)

''* Adding the [[YCoCg]] colorspace (with different sized planes) for RGB sourced pictures would be nice too. [[User:Elte|Elte]] 07:15, 16 March 2009 (EDT)

=== Extend GIF Encoder and Decoder to support Animated GIFs ===

=== Implement a Vivo demuxer ===
Implement a demuxer for the [[Vivo]] file format. The best reference for understanding the format would be MPlayer's [http://svn.mplayerhq.hu/mplayer/trunk/libmpdemux/demux_viv.c?view=markup existing .viv demuxer].

This task corresponds to ticket 132: https://avcodec.org/trac/ffmpeg/ticket/132

''I am ready to help out with understanding MPlayer's demuxer, esp. MPlayer API stuff if necessary.
--[[User:Reimar|Reimar]] 15:46, 1 March 2008 (EST)

=== Port missing demuxers from MPlayer ===
MPlayer supports a few container formats in libmpdemux that are not yet present in libavformat. Porting them over and gettting them relicensed as LGPL or reimplementing them from scratch should make reasonable small tasks.

# TiVo --
# VIVO -- ''Daniel Verkamp has a patch for this''
# SL support for MPEG-TS (anyone got samples?)
# MNG -- ''Paul B Mahol is working on this''

=== Optimal Huffman tables for (M)JPEG ===
This task is outlined at http://guru.multimedia.cx/small-tasks-for-ffmpeg/ and is tracked in the issue tracker: http://roundup.libav.org/issue267

=== M95 Playback System ===
This task is to implement a playback subsystem for [[M95]] files. This will entail writing a new file demuxer and video decoder (the audio is already uncompressed), both of which should be fairly easy by FFmpeg standards. [[M95|The M95 page]] contains the specs necessary to complete this task and points to downloadable samples.

=== BRP Playback System ===
This task is to implement a playback subsystem for [[BRP]] files. This will entail writing a new file demuxer as well as a video decoder that can handle at least 2 variations of format data. Further, write an audio decoder for the custom DPCM format in the file. All of these tasks are considered fairly easy by FFmpeg standards. [[BRP|The BRP page]] contains the specs necessary to complete this task and points to downloadable samples for both known variations.

=== 16-bit VQA Video Decoder ===
Westwood [[VQA]] files are already supported. However, there are three variations of its custom video codec, only the first two of which are supported. This task involves implementing support for the third variation. Visit the VQA samples repository: http://samples.multimedia.cx/game-formats/vqa/ -- The files in the directories Tiberian Sun VQAs/, bladerunner/, and dune2000/ use the 3rd variation of this codec. The [[VQA|VQA page]] should link to all the details you need to support this format.

Discussion/patch:
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/89902/focus=90433

=== HNM4 Playback System ===
This task is to implement a playback subsystem for [[HNM4]] variant of the [[HNM]] format. This will entail writing a new file demuxer and video decoder, both of which are considered fairly easy by FFmpeg standards. [[HNM4|The HNM4 page]] contains the specs necessary to complete this task and links to downloadable samples.

=== Apple RPZA encoder ===
A patch was once sent to the ffmpeg-devel mailing list to include an encoder for the [[Apple RPZA]] video codec. That code can be found on the "[[Interesting Patches]]" page. This qualification task involves applying that patch so that it can compile with current HEAD and then cleaning it up per the standards of the project. Engage the mailing list to learn more about what to do.

=== QuickTime Edit List Support ===
Implement edit list support in the QuickTime demuxer (libavformat/mov.c). This involves parsing the 'elst' atom in a QuickTime file. For a demonstration of how this is a problem, download the file menace00.mov from http://samples.mplayerhq.hu/mov/editlist/ and play it with ffplay or transcode it with ffmpeg. Notice that the audio and video are ever so slightly out of sync. Proper edit list support will solve that. Other samples in that directory also presumably exhibit edit list-related bugs. The [http://xine.cvs.sourceforge.net/xine/xine-lib/src/demuxers/demux_qt.c?view=markup Xine demuxer] has support for this, it might be useful for hints.

(patch was submitted to ffmpeg-devel , around 14 March 2009)

=== Add wma fixed point decoder back into libavcodec ===
http://svn.rockbox.org/viewvc.cgi/trunk/apps/codecs/libwma/
Rockbox's fixed-point WMA decoder was adapted from the decoder in libavcodec.

=== VC1 timestamps in m2ts ===

Codec copy of VC1 from m2ts currently doesn't work. Either extend the VC1 parser to output/fix timestamps, or fix the timestamps from m2ts demuxing.

=== flip flag for upside-down codecs ===

<pre>about the flip, a patch that decodes images fliped when
codec_tag == ff_get_fourcc("GEOX") is welcome.
its a metter of 2lines manipulating data/linesize of imgages after
get_buffer() or something similar
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
</pre>
more info:
http://roundup.ffmpeg.org/roundup/ffmpeg/issue741

=== lavf-based concatenation tool ===

It would be nice to have some libavformat-based tool that would extract frames from multiple files (possible different containers as well) and put them into single one.

=== vcr1 encoder ===
According to this: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-February/063555.html vcr1 encoder is disabled, and won't compile if enabled. Michael would prefer to keep it around, and have someone grow it into full encoder.

=== implement some colorspace fourcc/codecs ===
some colorspace formats were uploaded to http://samples.mplayerhq.hu/V-codecs/
including:
CYUV.AVI is 8 Bit Interleaved 4:2:2
a12v.avi is 4:2:2:4 10 Bit Interleaved
auv2.avi is 4:2:2:4 8 Bit Interleaved
and V-codecs/yuv8/MAILTEST.AVI .

it might decode with current pixfmts, a sample commit is 9853bbb21a19d540850de60d3e9cf7c6ef9da7dc

a sample commit for adding new input formats to swscale is 4884b9e50d416f84e64bfaf546a03e490cb83a2f
the hunks 3 and 5 you do not need, they are optional special converters
also the change to isSupportedOut() you do not need
above will add a new input format

another example for adding an input format is a43fb6b37efa5b01f2c9bdc414570691229bcfab

=== Implement Phantom Cine demuxer and Bayer format support for swscale ===
The format is described here:
http://wiki.multimedia.cx/index.php?title=Phantom_Cine
It will need support for Bayer -> RGB conversion in swscale to make the demuxer useful though.

=== support for [[YCoCg]]/RGB colorspace in FFV1 ===
Add support for [[YCoCg]] and [[RGB]] encoded sources for the [[FFV1]] codec

This would add a free lossless intra-frame RGB codec for all supported platforms (most important OS X + Windows) which is often asked for video editing in video forums (e.g. slashcam.de)

=== [[IFF#ANIM|IFF ANIM]] decoder ===
Modify libavformat/iff.c to handle this chunk and write a decoder for the format. The wiki page at [[IFF#ANIM|IFF ANIM]] has links to more information and source code. Samples can be found at http://www-user.tu-chemnitz.de/~womar/projects/iffanim/iffanim_samplepack.zip .

=== port missing decoders/demuxers from other open source projects. ===

http://www.mega-nerd.com/libsndfile/#Features
Paris Audio File PAF
IRCAM SF
GNU Octave 2.0 MAT4
GNU Octave 2.1 MAT5
Portable Voice Format PVFSound
Designer II SD2
samples are here: http://www.mega-nerd.com/tmp/SoundFileCollection-20050711-0902.tgz

http://www.hawksoft.com/hawkvoice/
HVDI_VOICE_DATA- packet
[[GSM]]
LPC
CELP
LPC10

http://sourceforge.net/projects/vgmstream
150+ formats: http://vgmstream.svn.sourceforge.net/viewvc/vgmstream/readme.txt

http://www.imagemagick.org
http://www.graphicsmagick.org/formats.html
many image formats not supported yet.

http://gpac.sourceforge.net/
[[MPEG-4 BIFS]]
3GPP DIMS
[[LASeR]]
SAF
SVG
[[Synchronized Multimedia Integration Language|SMIL]]
VRML
X3D
XMT

http://adplug.sourceforge.net/
http://adplug.sourceforge.net/library/
many OPL2/OPL3 audio formats not supported yet.

http://mikmod.raphnet.net/
http://mikmod.raphnet.net/#features
many music pattern formats not supported yet.

http://www.fly.net/~ant/libs/audio.html#Game_Music_Emu
AY
GBS
GYM
HES
KSS
NSF, NSFE
SAP
[[SNES-SPC700 Sound Format]]
VGM, VGZ

=== libswscale PAL8 output ===

See the thread: "[RFC] libswscale palette output implementation":
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/101397

=== vloopback output support ===

vloopback is a linux kernel device which allows to create a virtual video device where
programs can write, and can be accessed as a normal video device:
http://www.lavrsen.dk/twiki/bin/view/Motion/VideoFourLinuxLoopbackDevice

This would allow to write the ffmpeg output to a vloopdevice and be displayed by some a
program reading from such device (e.g. skype, a voip client etc.).

An example of a program which uses vloopback:
http://www.ws4gl.org/

=== Port video filters from MPlayer/VLC/Mjpegtools/Effectv/etc etc to libavfilter ===

There are plenty programs providing their own filters, many of them may be easily ported to the
superior ;-) framework of libavfilter. Also may be possible to create wrappers around other libraries
(e.g. opencv, libgimp, libshowphoto, libaa).

=== rar/zip/gz/bz2 etc demuxer ===
there are still compressed files out there (zipped raw wav, multi-rar'ed videos etc). create a decompression demuxer for them.

=== Less common AAC decoder features ===

Add support to the AAC decoder for object type ER AAC LC or AAC LC 960.

=== arithmetic decoding (and encoding) for mjpeg ===
Following marker codes are not supported by our mjpeg decoder:
DAC, SOF9, SOF10, SOF11, SOF13, SOF14 and SOF15.

=== [[Enhanced Variable Rate Codec]] decoder ===
samples and specs available.

=== adobe http f4f segmented fragmentation dynamic streaming format ===
sample streams on http://www.fox.com . command line instructions for creating such files: http://help.adobe.com/en_US/HTTPStreaming/1.0/Using/WS9463dbe8dbe45c4c-c126f3b1260533756d-7ffc.html . spec is available under adobe NDA. not to be confused with freely available F4V specification. open source php to convert f4f to flv: https://github.com/svnpenn/dotfiles/blob/master/etc/AdobeHDS.php

=== get 3IV1 decoder working and benchmark ===
we have a decoder for 3IV1. its currently if 0'd in the mpeg4 decoder. your task is to test if it still builds and works, and fix it so that it does not slow down mpeg4 decoder if enabled. the end goal is to enable it by default.

== Reverse Engineering Tasks ==

=== Demuxer for csf format and video codec ===
This is partially analyzed in http://ffmpeg.org/trac/ffmpeg/ticket/1060

=== realplayer's ivr format and create a demuxer for it ===
samples on [[IVR]] page.

=== pick a random binary codec from mplayer ===
[[MPlayer]] has over 100 binary codecs which have no opensource decoder. pick one, find a sample and try to reverse engineer it. note that some work has been done on some codecs, and its a good idea to ask on the mailing list before starting.

=== emblaze demuxer/decoder from java code ===
samples and java decoder: http://samples.mplayerhq.hu/internets/emblaze/

== Non-Programming Tasks ==

=== Check Linux distributions for patches to ffmpeg ===

check various distros like Fedora, Ubuntu, Debian, Mint, Arch, Suse etc for patches to ffmpeg. write down location of patches so it can be checked on an annual basis. if patches are found , report to ffmpeg-devel mailing list or bug trac.

=== improve layout and accessability of ffmpeg website ===

test ffmpeg.org with various browsers, including screen readers and get it optimized and available to people with poor vision. check wording and general ease of use. for example putting large download links for users like vlc and firefox have.

=== restore and update compatability page on ffmpeg website ===

we used to have a page that detailed how to create files for other software players and operating systems. restore this page from git history and update it for new devices and standardized codecs (h264 is the preferred codec now).

=== contact large ffmpeg users for broken / unplayable samples ===
contact the largest users of ffmpeg, like youtube, facebook, archive.org, blip.tv and others and ask them for access to samples that do not decode correctly.

=== review sample request error messages ===
ffmpeg has an av_log_ask_for_sample generic log message to ask the user for a sample when there is a problem. your task is to review ffmpeg decoders and demuxers (and possibly other inputs) and replace regular av_log messages requesting samples with it. example commit here: http://ffmpeg.org/pipermail/ffmpeg-cvslog/2011-April/036509.html

=== write bluray and 3d howtos ===
write a document or wiki article or just link to some info on how to play + encode + rip bluray using ffmpeg/mplayer and the various bluray libs required. also a guide on how to use ffmpeg/mplayer/vlc to encode and play various 3D formats. including maybe some supported hardware screens/video cards w/ examples. includes updating this wiki page [[Blu Ray and HD-DVD Playback Status]]
[[How to make a 3d movie with ffmpeg]]

=== write fate breakage notification bot ===
script up a bot to scan http://fate.ffmpeg.org that tells #ffmpeg-devel when something breaks compilation in fate.

=== write ipad/iphone/ios howto ===
write up some documentation on how to compile ffmpeg/ffplay/ffserver for iOS. exact tool versions, command lines, library requirements. both native compilation on the device and cross-compile using OS X. an android howto for various devices would be useful too.

[[Category:FFmpeg]]
[[Category:Libav]]

FFmpeg Summer of Code 2012

2012-03-27T19:02:52Z

Reimar:

FFmpeg Summer of Code Proposals and Qualification Tasks.

==Timeline==
March 12th-16th: Project application evaluation.

NOTE: FFmpeg has not been accepted this year, so unless you want to do any of these, the small tasks or qualification tasks just for fun...

==Contacting Developers==
Find us on irc, server: irc.freenode.net channel: #ffmpeg-devel
or contact us by subscribing to the mailing list https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel/

== Qualification Tasks ==
To be eligible for a Summer of Code project, we ask you to do a small programming task to prove you know the basics. FFmpeg is a large, complicated collection of code and it's not easy for beginners. There are some ideas for tasks on the [[Small FFmpeg Tasks]] page.

== 1st Tier Project Proposals ==
These are proposals with a mentor attached.

''Baptiste has also offered to mentor.''

=== hwaccel: global architecture ===

Revisit my older v2 proposal to completely get rid of HW pixel formats. The advantages were: this simplifies things, clean ups code, allows to move more code to the codec library (FFmpeg), allow for fallbacks to SW decoding (at the stream level if HW cannot meet the requirements, not while decoding a particular stream). The key point was to separate identification of the pixel format from the underlying HW accelerator. This will also allow for reading HW surfaces back, when requested if the selected pixel format.

Add encoding and post processing infrastructure.

* VA-API (hwaccel)

:Codecs related:
+ Add support {,M}JPEG decoding
+ Add support for VC-1 interlaced acceleration
+ Add support for H.264 interlaced

:Tools related:
:Add VA support to ffplay (see my older patches)
+ requires some VO infrastructure work
+ requires hwaccel hooks
+ requires enabling vaapi, dxva, bcm, tidsp, whatever

* Add VA support to ffmpeg
+ requires VA/DRI API, for no X dependency
+ dependencies: hwaccel v2 to construct pipelines (HW->HW, HW->SW, SW->HW, etc.), or at least allow for VA surface readback wherever necessary.

* VDPAU (hwaccel)

Update and push my older hwaccel-based VDPAU code. It should still work. :)
:Priority: hwaccel v2 infrastructure work first, ffplay enabling second.

''Mentor: Gwenole Beauchesne''

=== glplay ===
Add OpenGL output to ffplay, this should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which you may relicense under the LGPL.

''Mentor: Reimar Döffinger''

=== Improve the audio resampling/rematrixing/converting code ===

* Right now, we're using libswresample to resample/rematrix audio (samplerate / channels) and to resample the audio format (int, float, 16-bit, 32-bit).
* Both interleaved and planar audio sample formats are already supported
* We need SIMD optimization of popular conversions (float-int16, int16-float), (stereo-mono-5.1) and anything else that's frequently used.
* We need support for alternate conversion functions (e.g. sample format conversion with or without dithering)
* Fix bugs in current design (none known but there sure are some)

''Mentor: [[User:Michael|Michael Niedermayer]]''

=== Implement a H.265 / High efficiency video coding (HEVC) decoder ===

* Write a basic decoder supporting I, P, and, if time permits, B slices.
* It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.
* As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.
* A draft spec is available at: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/8_San%20Jose/wg11/JCTVC-H1003-v21.zip

''Mentor: [[User:Michael|Michael Niedermayer]]''

=== H.264 MVC ===

* Add MVC support to our H.264 decoder. MVC is used in 3D Blu-Rays.
* As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.

''Mentor: [[User:Michael|Michael Niedermayer]]''

=== Libavfilter extension ===

Libavfilter is the FFmpeg filtering library that started as a 2007 SoC [[FFmpeg Summer Of Code#Video Filter API (AKA libavfilter)|project]]. It currently supports audio and video filtering and generation support.

The task would consist of writing or porting audio and video filters and eventually fix/extend libavfilter API and design.

In particular the work may focus on porting MPlayer filters which are currently integrated through the mp wrapper.
For each port the student should verify that the new filter produces the same output (by comparing the output generated by -vf mp=FILTER and -vf FILTER) and checking that the new integrated filter is not slower.

Prerequisites: good C coding skills, familiarity with git/source code control systems, having some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

For getting more ideas read also the [[FFmpeg_/_Libav_Summer_Of_Code_2011#Libavfilter_video_work|GSoC 2011 libavfilter video proposal]] and [https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].

Qualification task: a port or a new implementation of one or more filters.

''Mentor: Stefano Sabatini - saste on IRC (possibly with a backup mentor)''.

=== Bayer colorspace formats ===

Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmepg's applicability to RAW still and video photography processing. Tasks:
* Implement bayer transformations in libswscale (plain C)
* Add bayer formats to the libavutil pixfmt enumeration routines
* Extend TIFF decoder to support DNG-Bayer format
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)
* SIMD optimizations of the libswscale transformations
* Decoders/specs may be available in the [[Dcraw]] project

Qualification task: TBD

''Mentor: [[User:Suxen_drol|Peter Ross]]''

=== Extend image formats support ===

Improve FFmpeg support for image formats, adding missing formats (e.g. XPM) and extending support for the current ones (e.g. animated GIF, GIF compression, fix PNG todos, add support to animated PNG) etc.

Qualification task: TBD (possibly finally fixing and integrating Måns' zlib decoder that has been unmerged since ages? Or just starting with some small part of the task itself, or implementing format autodetection for imagepipe demuxer)

''Mentor: [[User:reimar|Reimar Döffinger]]''

=== DTS-HD decoder ===

* ETSI released specifications (http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). Your job is to complete the existing decoder with the following features.

(1) Add support for mixed Core + DTS-HD stream structure
(DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main
and commentary tracks.
(2) Add support for XXCh extension (6.1 and 7.1 channels).
(3) Add support for X96 extension (96khz).
(4) Add support for XLL extension (lossless).
(5) Add support for a pure DTS-HD stream structure
(DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks.
(6) Add support for XBR extension (extra bitrate).

''Mentor: Benjamin Larsson''

=== Libavdevice API ===
Currently libavdevice use libavformat API. This is far from being useful for real-time video rendering. Your task is to write new API which resolves this and similar issues and port all already present input and output devices to the new API.

New API should have:
* Support for output video devices which do not provide own scaler.
* Minimal latency.

Qualification task: write one new video outdev, for example any of, but not limited to: aalib, caca, x11, xv, xover, vesa or gl2 using current libavdevice API.

''Mentor: [[User:Pbm|Paul B Mahol]]''

=== Redesigning the protocol reads ===
Eliminate active polling while allowing interrupts, and allowing selection on multiple handles.

''Mentor: Nicolas George''

== 2nd Tier (need mentor) Project Proposals ==
Some of the following proposals are also proposed by other organizations, we will try to coordinate this with them so as to avoid duplicate work.
We are also happy to hear your personal project ideas ...

=== AAC decoder improvements ===
Our AAC decoder does not support low-delay. Part of this task will be to also finish last year's BSAC task. A possible qualification task is to fix a crash in the current BSAC code with one of the samples from the BSAC testing suite.
:Sample: https://ffmpeg.org/trac/ffmpeg/ticket/113

=== AAC encoder improvements ===
Our AAC encoder does not produce competitive quality per bitrate.
Improve the encoder to be better than some other commonly used encoder like libfaac.
This requires solid understanding of things like psychoacoustics and rate distortion.
A qualification task for this could be to improve the encoder by at least 5% bitrate at the same quality
measured by some objective measure.

=== FF Fuzzer ===
Write a system like FATE that fuzzes and tests these fuzzed multimedia files under address sanitizer and valgrind with ffmpeg and ffplay.
When a crash, or other anomaly is found, it would use git bisect to identify which exact commit introduced the bug.
And either display this via some web frontend (similar to fate.ffmpeg.org) or just automatically send an email to some
dedicated mailing list. The system has to be robust (there will be infinite loops, OOM conditions and randomly occurring
crashes). Its also important that the system is easy to maintain and can filter out duplicates of the same issue.

=== VC1 interlaced ===
FFmpeg has code for interlaced VC1, but nearly all samples still do not decode correctly. The task is to finish last year's project. You should be able to find a possible qualification task by testing interlaced samples.

=== MPEG-4 ALS Roundup ===

This task is to update and enhance the existing ALS decoder as well as integrate
and enhance the rudimentary encoder found at:
https://github.com/justinruggles/FFmpeg-alsenc

Possible features are:

* implement rls-lms in the decoder
* do correct channel layout/sort handling in the decoder
* update to current master
* use codec private options
* implement encode2(), setting pts and duration
* document options and examples in encoders.texi
* come up with a good set of encoding tests for FATE
* implement mcc/channel sort in the encoder
* implement rls-lms in the encoder
* implement float support

=== Fix and improve FFserver ===
FFserver has been part of FFmpeg since a long time but due to lack of a motivated maintainer its a bit buggy.
For this project you would have to debug and fix many bugs. It requires good skills at reading and understanding other peoples code.
As a qualification task you will have to write functioning regression tests for FFserver which implicates some bugfixing to make ffserver produce the same output on all supported platforms.

=== Reverse engineer [[TAK]] codec and write decoder for it ===
TAK format is partially already documented.
You need to revisit that documentation and update and/or fix any missing/wrong information.
Reverse engineer codec and write working bitexact decoder.

=== Support for more subtitle formats ===
We have libass support now, either a parser (from mplayer) to convert subs into ass or something else.
* http://mailman.videolan.org/pipermail/vlc-devel/2011-September/081803.html
* http://ale5000.altervista.org/subtitles.htm

=== MKV ordered chapters / playlist support ===
Get playlist stuff into ffmpeg. Playlist is blocking a few things like quicktime edit list and .asx / .pls files.

=== Adobe fragmented http in/out ===
Adobe has a new streaming format.
* info/spec: http://www.adobe.com/products/httpdynamicstreaming/

=== libavfilter 9/10bit support ===
Make filters work with higher bitrate codecs/colorspaces

=== Fix copying video between formats ===
Lots of h264 streams in flv, mp4, mkv, mpegts. People wish to remux these into various formats so it works on their hardware (ps3, ipod etc).
https://ffmpeg.org/trac/ffmpeg/ticket/796
https://ffmpeg.org/trac/ffmpeg/ticket/822
https://ffmpeg.org/trac/ffmpeg/ticket/954
https://ffmpeg.org/trac/ffmpeg/ticket/976

=== Extend paletted format support ===

Cleanup framework for handling better with paletted format, write a posterize filter, add support to libswscale palette output (possibly making use of libavcodec/elbg), add support for reading and saving a palette to a file and apply them to the input video (e.g. by creating ad-hoc filters).

=== Port formats/colorspace support from dcraw or make dcraw wrapper ===
[[Dcraw]] supports many raw camera formats that ffmpeg may not. Port or make a wrapper for this project.

[[Category:FFmpeg]]

CSF

2012-03-14T16:44:36Z

Reimar:

Currently discussed and sample here: https://ffmpeg.org/trac/ffmpeg/ticket/1060
Copy-and-pasted description:
It is a format based on PNG and partial re-encoding, as you can see when you look at it with e.g. http://sourceforge.net/projects/extractor-gtk/
Possibly contains some other encoding types, too.
All container structures seem to be little-endian.
Each "frame" contains a 4 byte size at offset 8.
For the PNG encoded ones, offsets 16 - 24 seem to contain 4 16 bit values that indicate the offset and size of the image the PNG updates. After that seems to follow yet another size field containing the remaining bytes.
Start of the following packet is always aligned to 16 bytes.
There seems to be some kind of index at 0x190.
This and the data start seems to be referenced in the header, at bytes 24 and 28.

FFmpeg Summer of Code 2012

2012-03-11T08:52:26Z

Reimar: /* extend image formats support */

:Please detail the deadlines for applying and being a student/mentor.

FFmpeg Summer of Code Proposals and Qualification Tasks.

==Timeline==
March 12th-16th: Project application evaluation.

=Contacting Devels=
Find us on irc, server: irc.freenode.net channel: #ffmpeg-devel
or contact us by subscribing to the mailing list https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel/

== Qualification Tasks ==
To be eligible for a Summer of Code project, we ask you to do a small programming task to prove you know the basics. FFmpeg is a large, complicated collection of code and its not easy for beginners. There are some ideas for tasks on the [[Small FFmpeg Tasks]] page.

== 1st Tier Project Proposals ==
These are proposals with a mentor attached.

Baptiste has also offered to mentor.

=== Improve the audio resampling/rematrixing/converting code ===

* right now, we're using libswresample to resample/rematrix audio (samplerate / channels) and to resample the audio format (int, float, 16-bit, 32-bit).
* both interleaved and planar audio sample formats are already supported
* We need SIMD optimization of popular conversions (float-int16, int16-float), (stereo-mono-5.1) and anything else thats frequently used.
* We need support for alternate conversion functions (e.g. sample format conversion with or without dithering)
* fix bugs in current design (none known but there sure are some)

''Mentor: [[User:Michael|Michael Niedermayer]]''

=== Libavfilter extension ===

Libavfilter is the FFmpeg filtering library that started as a 2007 SoC [[FFmpeg Summer Of Code#Video Filter API (AKA libavfilter)|project]]. It currently supports audio and video filtering and generation support.

The task would consist into writing or porting audio and video filters and eventually fix/extend libavfilter API and design.

In particular the work may focus on porting MPlayer filters which are currently integrated through the mp wrapper.
For each port the student should verify that the new filter produces the same output (by comparing the output generated by -vf mp=FILTER and -vf FILTER) and checking that the new integrated filter is not slower.

Prerequisites: good C coding skills, familiarity with git/source code control systems, having some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

For getting more ideas read also the [[FFmpeg_/_Libav_Summer_Of_Code_2011#Libavfilter_video_work|GSoC 2011 libavfilter video proposal]] and [https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].

Qualification task: a port or a new implementation of one or more filters.

''Mentor: Stefano Sabatini - saste on IRC (possibly with a backup mentor)''.

=== Bayer colorspace formats ===

Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaded FFmepg's applicability to RAW still and video photography processing. Tasks:
* Implement bayer transformations in libswscale (plain C)
* Add bayer formats to the libavutil pixfmt enumeration routines
* Extend TIFF decoder to support DNG-Bayer format
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)
* SIMD optimisations of the libswscale transformations

Qualification task: TBD

''Mentor: [[User:Suxen_drol|Peter Ross]]''

=== extend image formats support ===

Improve FFmpeg support for image formats, adding missing formats (e.g. XPM) and extending support for the current ones (e.g. animated GIF, GIF compression, fix PNG todos, add support to animated PNG) etc.

Qualification task: TBD (possibly finally fixing and integrating Måns' zlib decoder that has been unmerged since ages? Or just starting with some small part of the task itself, or implementing format autodetection for imagepipe demuxer)

''Mentor: [[User:reimar|Reimar Döffinger]]''

== 2nd Tier (need mentor) Project Proposals ==
some following proposals might be done by libav, still waiting for dust to settle.

=== H.264 MVC ===
* Implement an H.264 MVC decoder. This format is used in 3D Blu-Rays.

=== DTS-HD decoder ===

* ETSI released specifcations (http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). Your job is to complete the existing decoder with the following features.

(1) Add support for mixed Core + DTS-HD stream structure
(DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main
and commentary tracks.
(2) Add support for XXCh extension (6.1 and 7.1 channels).
(3) Add support for X96 extension (96khz).
(4) Add support for XLL extension (lossless).
(5) Add support for a pure DTS-HD stream structure
(DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks.
(6) Add support for XBR extension (extra bitrate).

=== MPEG-4 ALS Roundup ===

This task is to update and enhance the existing ALS decoder as well as integrate
and enhance the rudimentary encoder found at:
https://github.com/justinruggles/FFmpeg-alsenc

Possible features are:

* implement rls-lms in the decoder
* do correct channel layout/sort handling in the decoder
* update to current master
* use codec private options
* implement encode2(), setting pts and duration
* document options and examples in encoders.texi
* come up with a good set of encoding tests for FATE
* implement mcc/channel sort in the encoder
* implement rls-lms in the encoder
* implement float support

=== reverse engineer [[TAK]] format ===
there is some interest in this format. maybe a qual task?

=== support for more subtitle formats ===
we have libass support now, either a parser (from mplayer) to convert subs into ass or something else.
* http://mailman.videolan.org/pipermail/vlc-devel/2011-September/081803.html
* http://ale5000.altervista.org/subtitles.htm

=== mkv ordered chapters / playlist support ===
get playlist stuff into ffmpeg. playlist is blocking a few things like quicktime edit list and .asx / .pls files.

=== adobe fragmented http in/out ===
adobe has a new streaming format.
* info/spec: http://www.adobe.com/products/httpdynamicstreaming/

=== libavfilter 9/10bit support ===
make filters work with higher bitrate codecs/colorspaces

=== fix copying video between formats ===
lots of h264 streams in flv, mp4, mkv, mpegts. people wish to remux these into various formats so it works on their hardware (ps3, ipod etc).
https://ffmpeg.org/trac/ffmpeg/ticket/796
https://ffmpeg.org/trac/ffmpeg/ticket/822
https://ffmpeg.org/trac/ffmpeg/ticket/954
https://ffmpeg.org/trac/ffmpeg/ticket/976

=== extend paletted format support ===

Cleanup framework for handling better with paletted format, write a posterize filter, add support to libswscale palette output (possibly making use of libavcodec/elbg), add support for reading and saving a palette to a file and apply them to the input video (e.g. by creating ad-hoc filters).

=== g2m4 decoder ===
lots of videolan users are requesting this codec be supported. there is a binary decoder in mplayer. its possible maxim is already working on this.

FFmpeg Summer of Code 2012

2012-03-11T08:51:35Z

Reimar:

:Please detail the deadlines for applying and being a student/mentor.

FFmpeg Summer of Code Proposals and Qualification Tasks.

==Timeline==
March 12th-16th: Project application evaluation.

=Contacting Devels=
Find us on irc, server: irc.freenode.net channel: #ffmpeg-devel
or contact us by subscribing to the mailing list https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel/

== Qualification Tasks ==
To be eligible for a Summer of Code project, we ask you to do a small programming task to prove you know the basics. FFmpeg is a large, complicated collection of code and its not easy for beginners. There are some ideas for tasks on the [[Small FFmpeg Tasks]] page.

== 1st Tier Project Proposals ==
These are proposals with a mentor attached.

Baptiste has also offered to mentor.

=== Improve the audio resampling/rematrixing/converting code ===

* right now, we're using libswresample to resample/rematrix audio (samplerate / channels) and to resample the audio format (int, float, 16-bit, 32-bit).
* both interleaved and planar audio sample formats are already supported
* We need SIMD optimization of popular conversions (float-int16, int16-float), (stereo-mono-5.1) and anything else thats frequently used.
* We need support for alternate conversion functions (e.g. sample format conversion with or without dithering)
* fix bugs in current design (none known but there sure are some)

''Mentor: [[User:Michael|Michael Niedermayer]]''

=== Libavfilter extension ===

Libavfilter is the FFmpeg filtering library that started as a 2007 SoC [[FFmpeg Summer Of Code#Video Filter API (AKA libavfilter)|project]]. It currently supports audio and video filtering and generation support.

The task would consist into writing or porting audio and video filters and eventually fix/extend libavfilter API and design.

In particular the work may focus on porting MPlayer filters which are currently integrated through the mp wrapper.
For each port the student should verify that the new filter produces the same output (by comparing the output generated by -vf mp=FILTER and -vf FILTER) and checking that the new integrated filter is not slower.

Prerequisites: good C coding skills, familiarity with git/source code control systems, having some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

For getting more ideas read also the [[FFmpeg_/_Libav_Summer_Of_Code_2011#Libavfilter_video_work|GSoC 2011 libavfilter video proposal]] and [https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].

Qualification task: a port or a new implementation of one or more filters.

''Mentor: Stefano Sabatini - saste on IRC (possibly with a backup mentor)''.

=== Bayer colorspace formats ===

Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaded FFmepg's applicability to RAW still and video photography processing. Tasks:
* Implement bayer transformations in libswscale (plain C)
* Add bayer formats to the libavutil pixfmt enumeration routines
* Extend TIFF decoder to support DNG-Bayer format
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)
* SIMD optimisations of the libswscale transformations

Qualification task: TBD

''Mentor: [[User:Suxen_drol|Peter Ross]]''

=== extend image formats support ===

Improve FFmpeg support for image formats, adding missing formats (e.g. XPM) and extending support for the current ones (e.g. animated GIF, GIF compression, fix PNG todos, add support to animated PNG) etc.

Qualification task: TBD (possibly finally fixing and integrating Måns' zlib decoder that has been unmerged since ages? Or just starting with some small part of the task itself)

''Mentor: [[User:reimar|Reimar Döffinger]]''

== 2nd Tier (need mentor) Project Proposals ==
some following proposals might be done by libav, still waiting for dust to settle.

=== H.264 MVC ===
* Implement an H.264 MVC decoder. This format is used in 3D Blu-Rays.

=== DTS-HD decoder ===

* ETSI released specifcations (http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). Your job is to complete the existing decoder with the following features.

(1) Add support for mixed Core + DTS-HD stream structure
(DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main
and commentary tracks.
(2) Add support for XXCh extension (6.1 and 7.1 channels).
(3) Add support for X96 extension (96khz).
(4) Add support for XLL extension (lossless).
(5) Add support for a pure DTS-HD stream structure
(DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks.
(6) Add support for XBR extension (extra bitrate).

=== MPEG-4 ALS Roundup ===

This task is to update and enhance the existing ALS decoder as well as integrate
and enhance the rudimentary encoder found at:
https://github.com/justinruggles/FFmpeg-alsenc

Possible features are:

* implement rls-lms in the decoder
* do correct channel layout/sort handling in the decoder
* update to current master
* use codec private options
* implement encode2(), setting pts and duration
* document options and examples in encoders.texi
* come up with a good set of encoding tests for FATE
* implement mcc/channel sort in the encoder
* implement rls-lms in the encoder
* implement float support

=== reverse engineer [[TAK]] format ===
there is some interest in this format. maybe a qual task?

=== support for more subtitle formats ===
we have libass support now, either a parser (from mplayer) to convert subs into ass or something else.
* http://mailman.videolan.org/pipermail/vlc-devel/2011-September/081803.html
* http://ale5000.altervista.org/subtitles.htm

=== mkv ordered chapters / playlist support ===
get playlist stuff into ffmpeg. playlist is blocking a few things like quicktime edit list and .asx / .pls files.

=== adobe fragmented http in/out ===
adobe has a new streaming format.
* info/spec: http://www.adobe.com/products/httpdynamicstreaming/

=== libavfilter 9/10bit support ===
make filters work with higher bitrate codecs/colorspaces

=== fix copying video between formats ===
lots of h264 streams in flv, mp4, mkv, mpegts. people wish to remux these into various formats so it works on their hardware (ps3, ipod etc).
https://ffmpeg.org/trac/ffmpeg/ticket/796
https://ffmpeg.org/trac/ffmpeg/ticket/822
https://ffmpeg.org/trac/ffmpeg/ticket/954
https://ffmpeg.org/trac/ffmpeg/ticket/976

=== extend paletted format support ===

Cleanup framework for handling better with paletted format, write a posterize filter, add support to libswscale palette output (possibly making use of libavcodec/elbg), add support for reading and saving a palette to a file and apply them to the input video (e.g. by creating ad-hoc filters).

=== g2m4 decoder ===
lots of videolan users are requesting this codec be supported. there is a binary decoder in mplayer. its possible maxim is already working on this.

PMP

2010-11-04T20:41:40Z

Reimar:

PMP format

All values are stored as little-endian

header:
0-3 "pmpm"
4-7 1 (version?)
8-b video format (0 = MPEG-4 ASP, 1 = H.264)
d-f number of packets (== number of frames?)
10-13 video width
14-17 video height
18-1b time base num
1c-1f time base den
20-23 audio format (0 = MP3, 1 = AAC)
24-27 number of audio streams (all the same format)
28-2b ??
2c-2f ??
30-33 sample rate
34-37 channels - 1?
38- list of packet sizes, 4 bytes each, lowest bit is keyframe flag
following: data packets

data packet:
0 number of audio packets (per audio stream)
1-4 ?
5-8 ?
9-c length of video data
audio packet sizes, 4 bytes each
video data
audio data, AAC packets lack 7 bytes of AAC header

PMP

2010-11-04T20:40:59Z

Reimar:

GSM 06.10

2010-07-03T22:15:25Z

Reimar:

* FOURCCs: agsm
* Website: http://kbs.cs.tu-berlin.de/~jutta/toast.html
* Specification: http://www.3gpp.org/ftp/Specs/2010-06/R1999/06_series/0610-820.zip

GSM 06.10 is a GSM [[vocoder]] standard that also occurs in some multimedia files. Also see [[Microsoft GSM]].

[[Category:Vocoders]]
[[Category:Audio Codecs]]
[[Category: Formats missing in FFmpeg]]

Format:
Each frame consists of 33 bytes, in a little-endian bitstream.

Bitstream layout:

4 bits: magic 0xd,
2 x 6 bits, 2 x 5 bits, 2 x 4 bits, 2 x 3 bits

4 x coefficients (
7 bits,
2 bits,
2 bits,
6 bits,
13 x 3 bits
)

GSM 06.10

2010-07-03T21:07:10Z

Reimar:

* FOURCCs: agsm
* Website: http://kbs.cs.tu-berlin.de/~jutta/toast.html
* Specification: http://www.3gpp.org/ftp/Specs/2010-06/R1999/06_series/0610-820.zip

GSM 06.10 is a GSM [[vocoder]] standard that also occurs in some multimedia files. Also see [[Microsoft GSM]].

[[Category:Vocoders]]
[[Category:Audio Codecs]]
[[Category: Formats missing in FFmpeg]]

Format:
Each frame consists of 33 bytes, in a bigendian bitstream.

Bitstream layout:

4 bits: magic 0xd,
2 x 6 bits, 2 x 5 bits, 2 x 4 bits, 2 x 3 bits

4 x coefficients (
7 bits,
2 bits,
2 bits,
6 bits,
13 x 3 bits
)

GSM 06.10

2010-07-03T11:54:10Z

Reimar:

* FOURCCs: agsm
* Website: http://kbs.cs.tu-berlin.de/~jutta/toast.html
* Specification: http://www.3gpp.org/ftp/Specs/2010-06/R1999/06_series/0610-820.zip

GSM 06.10 is a GSM [[vocoder]] standard that also occurs in some multimedia files. Also see [[Microsoft GSM]].

[[Category:Vocoders]]
[[Category:Audio Codecs]]
[[Category: Formats missing in FFmpeg]]

Format:
Each frame consists of 33 bytes, in a bigendian bitstream.

Bitstream layout:

4 bits: magic 0xd
8 x 6 bits

4 x coefficients (
7 bits
2 bits
2 bits
6 bits
13 x 3 bits
)

Phantom Cine

2010-06-13T12:12:50Z

Reimar: Clarify: current description is not based on specification

Extensions: cine
* Company: [[Vision Research Inc.]]
* Samples: [http://samples.mplayerhq.hu/V-codecs/Phantom_Cine http://samples.mplayerhq.hu/V-codecs/Phantom_Cine]

The Phantom Cine format

Note: below information is based on looking at a sample file, please verify against the specification linked at the end.

This is a raw uncompressed format.

It contains a lot of metadata, and index and raw uncompressed frames, presumably in Bayer format.
Know samples store each Bayer sample with 16 bits, where only the lowest 14 bits are relevant (the value 16 can be determined from the BitmapInfoHeader, the 14 bits are somewhere in the file as well, as can be seen from the xml-representation of the metadata (the RealBPP entry) - all data in the xml file exists in the .cine file as well). Making the lowest instead of the highest 14 bits the relevant ones is a rather unfortunate choice since it makes conversion to standard formats more difficult.

Note: some of the fixed offsets below might not always be at that offset, though they are for the known samples and there are no length fields that would allow calculating them in a sensible way.

All values are stored in little-endian format.

The files start with the bytes "CI", followed by two bytes possibly giving a size or index (all files so far have the value 2c 00).

This "main header" contains this data:
2 bytes "CI"
2 bytes length
2 bytes "compression type" (always 2) ??
2 bytes version (always 1)
4 bytes number of first frame (usually negative)
4 bytes number of frames in files
4 bytes number of first frame (repeated?)
4 bytes number of frames in files (repeated?)
4 bytes offset of BitampInfoHeader
4 bytes offset of ???
4 bytes offset of file index
8 bytes recording time stamp (?)

At offset 0x2c there is a BitmapInfoHeader structure. The first 4 bytes give its length (0x28).

The next sections starts with "ST" at offset 0xe0, again followed by two bytes giving a size or index (so far always 6c 16).

Variable length part seems to start at offset 0x16c0. Note that this is the sum of the possible length files from CI, ST and BitmapInfoHeader (i.e. 0x2c + 0x28 + 0x166c = 0x16c0).

It consists of 3 parts:

1) timestamps

2) exposure values

3) index

The timestamp and exposure values parts start with a 4-byte length field allowing to skip them.

The index contains a 8-byte offset value for each frame. It does not have a length field, the number of entries is determined from the number of frames in the main header.

At each offset indicated by the index, there is first a header. The first 4 bytes of the header give the length of the header. Only the value 8 has a known meaning, in which case the next 4 bytes give the length of the frame data.

== External Links ==
* [http://www.visionresearch.com/devzonedownloads/cine640.pdf http://www.visionresearch.com/devzonedownloads/cine640.pdf] The CINE File Format, Vision Research Inc., 2007

[[Category:Container Formats]]

Phantom Cine

2010-06-13T12:11:24Z

Reimar: typo fix

Extensions: cine
* Company: [[Vision Research Inc.]]
* Samples: [http://samples.mplayerhq.hu/V-codecs/Phantom_Cine http://samples.mplayerhq.hu/V-codecs/Phantom_Cine]

The Phantom Cine format

This is a raw uncompressed format.

It contains a lot of metadata, and index and raw uncompressed frames, presumably in Bayer format.
Know samples store each Bayer sample with 16 bits, where only the lowest 14 bits are relevant (the value 16 can be determined from the BitmapInfoHeader, the 14 bits are somewhere in the file as well, as can be seen from the xml-representation of the metadata (the RealBPP entry) - all data in the xml file exists in the .cine file as well). Making the lowest instead of the highest 14 bits the relevant ones is a rather unfortunate choice since it makes conversion to standard formats more difficult.

Note: some of the fixed offsets below might not always be at that offset, though they are for the known samples and there are no length fields that would allow calculating them in a sensible way.

All values are stored in little-endian format.

The files start with the bytes "CI", followed by two bytes possibly giving a size or index (all files so far have the value 2c 00).

This "main header" contains this data:
2 bytes "CI"
2 bytes length
2 bytes "compression type" (always 2) ??
2 bytes version (always 1)
4 bytes number of first frame (usually negative)
4 bytes number of frames in files
4 bytes number of first frame (repeated?)
4 bytes number of frames in files (repeated?)
4 bytes offset of BitampInfoHeader
4 bytes offset of ???
4 bytes offset of file index
8 bytes recording time stamp (?)

At offset 0x2c there is a BitmapInfoHeader structure. The first 4 bytes give its length (0x28).

The next sections starts with "ST" at offset 0xe0, again followed by two bytes giving a size or index (so far always 6c 16).

Variable length part seems to start at offset 0x16c0. Note that this is the sum of the possible length files from CI, ST and BitmapInfoHeader (i.e. 0x2c + 0x28 + 0x166c = 0x16c0).

It consists of 3 parts:

1) timestamps

2) exposure values

3) index

The timestamp and exposure values parts start with a 4-byte length field allowing to skip them.

The index contains a 8-byte offset value for each frame. It does not have a length field, the number of entries is determined from the number of frames in the main header.

At each offset indicated by the index, there is first a header. The first 4 bytes of the header give the length of the header. Only the value 8 has a known meaning, in which case the next 4 bytes give the length of the frame data.

== External Links ==
* [http://www.visionresearch.com/devzonedownloads/cine640.pdf http://www.visionresearch.com/devzonedownloads/cine640.pdf] The CINE File Format, Vision Research Inc., 2007

[[Category:Container Formats]]

Small FFmpeg Tasks

2010-03-28T12:13:14Z

Reimar: /* implement some colorspace fourcc/codecs */

This page contains ideas for small, relatively simple tasks for the [[FFmpeg]] project. People who might be interested in trying one of these tasks:
* Someone who wants to contribute to FFmpeg and needs to find a well-defined task to start with
* Someone who wishes to qualify for one of FFmpeg's coveted [[FFmpeg Summer Of Code|Summer of Code]] project slots
* An existing FFmpeg developer who has been away from the project for a while and needs a smaller task as motivation for re-learning the codebase

For other tasks of varying difficulty, see the [[Interesting Patches]] page.

'''If you would like to work on one of these tasks''', please take these steps:
* Subscribe to the [https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel FFmpeg development mailing list] and indicate your interest
* Ask [[User:Multimedia Mike|Multimedia Mike]] for a Wiki account so you can claim your task on this Wiki

'''If you would like to add to this list''', please be prepared to explain some useful details about the task. Excessively vague tasks with no supporting details will be ruthlessly deleted.

=== Finish up a previous incomplete SoC project ===

Several SoC projects from previous years have not yet made it into FFmpeg. Taking any of them and finishing them up to the point that they can be included should make for a good qualification task. Check out the [[FFmpeg Summer Of Code]] overview page and look for the unfinished projects, like AMR-NB, Dirac, TS muxer, JPEG 2000.

=== Generic Colorspace system ===
This task involves adding support more than 8 bits per component (Y on 10 bits, U on 10 bits, V on 10 bits for example)
and generic simple conversion to other colorspaces.

''Does this have to do with revising FFmpeg's infrastructure? If so, then it doesn't feel like a qualification task. If it's something simpler, then the vague description does not convey that simplicity. Please expound.'' --[[User:Multimedia Mike|Multimedia Mike]] 12:56, 25 February 2008 (EST)

''I don't think so, extending PixFmt to extended structure with finegrained description like depth, range values, colorspace, sample period, and write generic simple conversion from all formats to all others, like suggested by Michael on the mailing list. Conversion routine can be a good qualification task for video encoders/decoders. What do you think ?
--[[User:Bcoudurier|Baptiste Coudurier]] 00:30, 29 February 2008 (EST)

''* Adding the [[YCoCg]] colorspace (with different sized planes) for RGB sourced pictures would be nice too. [[User:Elte|Elte]] 07:15, 16 March 2009 (EDT)

=== Make the SoC dts encoder multichannel capable ===
Here is a skeleton for a dts encoder http://svn.mplayerhq.hu/soc/dcaenc/, currently it can only encode stereo streams.
The task is to extend it to support 5.1 channels also.

Specs and info can be found here:
http://wiki.multimedia.cx/index.php?title=DTS

=== GIF LZW Encoder and extend Encoder and Decoder to support Animated GIFs ===

Lzw encoder is already used for TIFF, it must be extended to support GIF flavor.

=== Implement a Vivo demuxer for FFmpeg ===
Implement an FFmpeg demuxer for the [[Vivo]] file format. The best reference for understanding the format would be MPlayer's [http://svn.mplayerhq.hu/mplayer/trunk/libmpdemux/demux_viv.c?view=markup existing .viv demuxer].

This task corresponds to issue 99: http://roundup.ffmpeg.org/roundup/ffmpeg/issue99

''I am ready to help out with understanding MPlayer's demuxer, esp. MPlayer API stuff if necessary.
--[[User:Reimar|Reimar]] 15:46, 1 March 2008 (EST)

=== Port missing demuxers from MPlayer to FFmpeg ===
MPlayer supports a few container formats in libmpdemux that are not yet present in libavformat. Porting them over and gettting them relicensed as LGPL or reimplementing them from scratch should make reasonable small tasks.

# TiVo -- ''Jai Menon is working on this''
# VIVO -- ''Daniel Verkamp has a patch for this''
# SL support for MPEG-TS (anyone got samples?)
# MNG

=== Optimal Huffman tables for (M)JPEG ===
This task is outlined at http://guru.multimedia.cx/small-tasks-for-ffmpeg/ and is tracked in the issue tracker: http://roundup.ffmpeg.org/roundup/ffmpeg/issue267

=== YOP Playback System ===
This task is to implement an FFmpeg playback subsystem for [[Psygnosis YOP]] files. This will entail writing a new file demuxer and video decoder, both of which are trivial by FFmpeg standards. [[Psygnosis YOP|The Psygnosis YOP page]] contains the specs necessary to complete this task and points to downloadable samples.
:''Patch pending on -devel

=== M95 Playback System ===
This task is to implement an FFmpeg playback subsystem for [[M95]] files. This will entail writing a new file demuxer and video decoder (the audio is already uncompressed), both of which are trivial by FFmpeg standards. [[M95|The M95 page]] contains the specs necessary to complete this task and points to downloadable samples.

=== BRP Playback System ===
This task is to implement an FFmpeg playback subsystem for [[BRP]] files. This will entail writing a new file demuxer as well as a video decoder that can handle at least 2 variations of format data. Further, write an audio decoder for the custom DPCM format in the file. All of these tasks are considered trivial by FFmpeg standards. [[BRP|The BRP page]] contains the specs necessary to complete this task and points to downloadable samples for both known variations.

=== 16-bit Interplay Video Decoder ===
FFmpeg already supports [[Interplay MVE]] files with [[Interplay Video|8-bit video data]] inside. This task involves supporting 16-bit video data. The video encoding format is mostly the same but the pixel size is twice as large. Engage the ffmpeg-devel list to discuss how best to approach this task.

=== 16-bit VQA Video Decoder ===
FFmpeg already supports Westwood [[VQA]] files. However, there are 3 variations of its custom video codec. The first 2 are supported in FFmpeg. This task involves implementing support for the 3rd variation. Visit the VQA samples repository: http://samples.mplayerhq.hu/game-formats/vqa/ -- The files in the directories Tiberian Sun VQAs/, bladerunner/, and dune2000/ use the 3rd variation of this codec. The [[VQA|VQA page]] should link to all the details you need to support this format.

Discussion/patch: ([http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-March/065348.html reference])

=== HNM4 Playback System ===
This task is to implement an FFmpeg playback subsystem for [[HNM4]] variant of the [[HNM]] format. This will entail writing a new file demuxer and video decoder, both of which are trivial by FFmpeg standards. [[HNM4|The HNM4 page]] contains the specs necessary to complete this task and links to downloadable samples.

=== Apple RPZA encoder ===
A patch was once sent to the ffmpeg-devel mailing list to include an encoder for the [[Apple RPZA]] video codec. That code can be found on the "[[Interesting Patches]]" page. This qualification task involves applying that patch so that it can compile with current FFmpeg SVN code and then cleaning it up per the standards of the project. Engage the mailing list to learn more about what to do.
:''Claimed by Jai Menon''

=== QuickTime Edit List Support ===
Implement edit list support in FFmpeg's QuickTime demuxer (libavformat/mov.c). This involves parsing the 'elst' atom in a QuickTime file. For a demonstration of how this is a problem, download the file menace00.mov from http://samples.mplayerhq.hu/mov/editlist/ and play it with ffplay or transcode it with ffmpeg. Notice that the audio and video are ever so slightly out of sync. Proper edit list support will solve that. Other samples in that directory also presumably exhibit edit list-related bugs. The [http://xine.cvs.sourceforge.net/xine/xine-lib/src/demuxers/demux_qt.c?view=markup Xine demuxer] has support for this, it might be useful for hints.

(patch was submitted to ffmpeg-devel , around 14 March 2009)

=== Implement the Flash Screen Video codec version 2 ===
FFmpeg is missing both a decoder and an encoder. Would be nice to have that.

:''Daniel Verkamp is working on this''

=== Add wma fixed point decoder back into libavcodec ===
http://svn.rockbox.org/viewvc.cgi/trunk/apps/codecs/libwma/
Rockbox's fixed-point WMA decoder was adapted from the decoder in libavcodec.

=== RealAudio 14.4 encoder ===
FFmpeg contains a decoder for [[RealAudio 14.4]], a farily simple integer CELP codec. Write an encoder. This would be a good qualification task for anyone interested in working on AMR, Speex, or sipr.

=== VC1 timestamps in m2ts ===

Codec copy of VC1 from m2ts currently doesn't work. Either extend the VC1 parser to output/fix timestamps, or fix the timestamps from m2ts demuxing.

=== FLIC work ===

Revise the [[Flic Video]] decoder at libavcodec/flicvideo.c to support video transported in AVI or MOV files while making sure that data coming from the usual FLI files still works. 'AFLC' and 'flic' FourCC samples are linked from the [[Flic Video]] page.

=== CJPG format ===

Extend FFmpeg's MJPEG decoder to handle the different frames/packing of CJPG. Samples at: http://roundup.ffmpeg.org/roundup/ffmpeg/issue777

=== flip flag for upside-down codecs ===

<pre>about the flip, a patch that decodes images fliped when
codec_tag == ff_get_fourcc("GEOX") is welcome.
its a metter of 2lines manipulating data/linesize of imgages after
get_buffer() or something similar
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
</pre>
more info:
http://roundup.ffmpeg.org/roundup/ffmpeg/issue741

=== lavf-based concatenation tool ===

Unless we have multiple files input in FFmpeg, it would be nice to have some libavformat-based tool that would extract frames from multiple files (possible different containers as well) and put them into single one.

=== cljr and vcr1 encoders ===
According to this: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-February/063647.html both of the encoders are disabled, and won't compile if enabled. Michael would prefer to keep them around, and have someone grow them into full encoders.

=== implement some colorspace fourcc/codecs ===
some colorspace formats were uploaded to http://samples.mplayerhq.hu/V-codecs/
including:
CYUV.AVI is 8 Bit Interleaved 4:2:2
a12v.avi is 4:2:2:4 10 Bit Interleaved
auv2.avi is 4:2:2:4 8 Bit Interleaved
and V-codecs/yuv8/MAILTEST.AVI .

it might decode with current pixfmts, for that all you will need is:
cd ffmpeg
svn di -r20378:20379

step by step tutorial for adding new input formats to swscale:
cd mplayer/libswscale/
svn di -r20426:20427
the hunks 3 and 5 you dont need, they are optional special converters
also the change to isSupportedOut() you dont need
above will add a new input format

another example for adding an input format
cd mplayer/libswscale/
svn di -r20604:20605

=== Implement Phantom Cine demuxer and Bayer format support for swscale ===
The format is described here:
http://wiki.multimedia.cx/index.php?title=Phantom_Cine
It will need support for Bayer -> RGB conversion in swscale to make the demuxer useful though.

=== Make the rtp demuxer support rtcp BYE packets ===
rtcp BYE (203) packets are sent from the sender to the receiver to notify that a stream has ended.
FFmpeg currently ignores them.

Sample url rtsp://media.lscube.org/tests/tc.mov

=== support for [[YCoCg]]/RGB colorspace in FFV1 ===
Add support for [[YCoCg]] and [[RGB]] encoded sources for the [[FFV1]] codec

This would add a free lossless intra-frame RGB codec for all by FFmpeg supported platforms (most important MacOS + Windows) which is often asked for video editing in video forums (e.g. slashcam.de)

=== Metal Gear Solid Video format demuxer ===
Write a demuxer to play video files harvested from the game Metal Gear Solid: The Twin Snakes. The format is described on the wiki page [[Metal Gear Solid VP3]] (which also contains links to samples). This page is based on observations and conjecture, so remember to engage the ffmpeg-devel mailing list with questions.

=== [[IFF#ANIM|IFF ANIM]] decoder ===
Modify libavformat/iff.c to handle this chunk and write a decoder for the format. The wiki page at [[IFF#ANIM|IFF ANIM]] has links to more information and source code. Samples can be found at http://www-user.tu-chemnitz.de/~womar/projects/iffanim/iffanim_samplepack.zip .

=== [[CDXL]] decoder ===
http://roundup.ffmpeg.org/roundup/ffmpeg/issue1012

Write a decoder for this format using the information in the [[CDXL]] wiki page
Discussed for the 2009 SoC

=== port missing decoders/demuxers from other open source projects. ===

http://www.mega-nerd.com/libsndfile/#Features
Paris Audio File PAF
IRCAM SF
GNU Octave 2.0 MAT4
GNU Octave 2.1 MAT5
Portable Voice Format PVFSound
Designer II SD2
samples are here: http://www.mega-nerd.com/tmp/SoundFileCollection-20050711-0902.tgz

http://www.hawksoft.com/hawkvoice/
HVDI_VOICE_DATA- packet
[[GSM]]
LPC
CELP
LPC10

http://sourceforge.net/projects/vgmstream
150+ formats: http://vgmstream.svn.sourceforge.net/viewvc/vgmstream/readme.txt

http://www.imagemagick.org
http://www.graphicsmagick.org/formats.html
many image formats not in ffmpeg yet.

http://gpac.sourceforge.net/
[[MPEG-4 BIFS]]
3GPP DIMS
[[LASeR]]
SAF
SVG
[[Synchronized Multimedia Integration Language|SMIL]]
VRML
X3D
XMT

http://adplug.sourceforge.net/
http://adplug.sourceforge.net/library/
many OPL2/OPL3 audio formats not in ffmpeg yet.

http://mikmod.raphnet.net/
http://mikmod.raphnet.net/#features
many music pattern formats not in ffmpeg yet.

http://www.fly.net/~ant/libs/audio.html#Game_Music_Emu
AY
GBS
GYM
HES
KSS
NSF, NSFE
SAP
[[SNES-SPC700 Sound Format]]
VGM, VGZ

=== port [[Ut Video]] decoder/encoder ===
gpl v2 decoder/encoder at wiki page

Claimed by shankhs ch as a GSoC 2010 qualification project.

=== Sony psp demuxer ===
create/port a demuxer for the sony playstation portable format PMP.
*Samples: http://samples.mplayerhq.hu/playstation/psp/
*mplayer demuxer: http://mplayer-ww.svn.sourceforge.net/viewvc/mplayer-ww/trunk/mplayer/libmpdemux/demux_pmp.c

=== libswscale PAL8 output ===

See the thread: "[RFC] libswscale palette output implementation":
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/101397

=== vloopback output support ===

vloopback is a linux kernel device which allows to create a virtual video device where
programs can write, and can be accessed as a normal video device:
http://www.lavrsen.dk/twiki/bin/view/Motion/VideoFourLinuxLoopbackDevice

This would allow to write the ffmpeg output to a vloopdevice and be displayed by some a
program reading from such device (e.g. skype, a voip client etc.).

An example of a program which uses vloopback:
http://www.ws4gl.org/

=== Port video filters from MPlayer/VLC/Mjpegtools/Effectv/etc etc to libavfilter ===

There are plenty programs providing their own filters, many of them may be easily ported to the
superior ;-) framework of libavfilter. Also may be possible to create wrappers around other libraries
(e.g. opencv, libgimp, libshowphoto, libaa).

=== Add weighted motion compensation for B-frames in RV3/4 ===

RealVideo 3 and 4 uses weighted motion compensation for B-frames (like H.264) but current code
always uses simple averaging. Reverse-engineer weighted MC functions for B-frames and make
RV3/4 decoder use them.

[[Category:FFmpeg]]

Phantom Cine

2010-03-28T12:09:31Z

Reimar:

The Phantom Cine format

This is a raw uncompressed format.

It contains a lot of metadata, and index and raw uncompressed frames, presumably in Bayer format.
Know samples store each Bayer sample with 16 bits, where only the lowest 14 bits are relevant (the value 16 can be determined from the BitmapInfoHeader, the 14 bits are somewhere in the file as well, as can be seen from the xml-representation of the metadata (the RealBPP entry) - all data in the xml file exists in the .cine file as well). Making the lowest instead of the highest 14 bits the relevant ones is a rather unfortunate choice since it makes conversion to standard formats more difficult.

Note: some of the fixed offsets below might not always be at that offset, though they are for the known samples and there are no length fields that would allow calculating them in a sensible way.

All values are stored in little-endian format.

The files start with the bytes "CI", followed by two bytes possibly giving a size or index (all files so far have the value 2c 00).

This "main header" contains this data:
2 bytes "CI"
2 bytes length
2 bytes "compression type" (always 2) ??
2 bytes version (always 1)
4 bytes number of first frame (usually negative)
4 bytes number of frames in files
4 bytes number of first frame (repeated?)
4 bytes number of frames in files (repeated?)
4 bytes offset of BitampInfoHeader
4 bytes offset of ???
4 bytes offset of file index
8 bytes recording time stamp (?)

At offset 0x2c there is a BitmapInfoHeader structure. The first 4 bytes give its length (0x28).

The next sections starts with "ST" at offset 0xe0, again followed by two bytes giving a size or index (so far always 6c 16).

Variable length part seems to start at offset 0x16c0. Note that this is the sum of the possible length files from CI, ST and BitmapInfoHeader (i.e. 0x2c + 0x28 + 0x166c = 0x16c0).

It consists of 3 parts:

1) timestamps

2) exposure values

3) index

The timestamp and exposure values parts start with a 4-byte length field allowing to skip them.

The index contains a 8-byte offset value for each frame. It does not have a length field, the number of entries is determined from the number of frames in the main header.

At each offset indicated by the index, there is first a header. The first 4 bytes of the header give the length of the header. Only the value 8 has a known meaning, in which case the next 4 bytes give the length of the frame data.

Phantom Cine

2010-03-28T12:02:35Z

Reimar:

The Phantom Cine format

This is a raw uncompressed format.

It contains a lot of metadata, and index and raw uncompressed frames, presumably in Bayer format.

Note: some of the fixed offsets below might not always be at that offset, though they are for the known samples and there are no length fields that would allow calculating them in a sensible way.

All values are stored in little-endian format.

The files start with the bytes "CI", followed by two bytes possibly giving a size or index (all files so far have the value 2c 00).

This "main header" contains this data:
2 bytes "CI"
2 bytes length
2 bytes "compression type" (always 2) ??
2 bytes version (always 1)
4 bytes number of first frame (usually negative)
4 bytes number of frames in files
4 bytes number of first frame (repeated?)
4 bytes number of frames in files (repeated?)
4 bytes offset of BitampInfoHeader
4 bytes offset of ???
4 bytes offset of file index
8 bytes recording time stamp (?)

At offset 0x2c there is a BitmapInfoHeader structure. The first 4 bytes give its length (0x28).

The next sections starts with "ST" at offset 0xe0, again followed by two bytes giving a size or index (so far always 6c 16).

Variable length part seems to start at offset 0x16c0. Note that this is the sum of the possible length files from CI, ST and BitmapInfoHeader (i.e. 0x2c + 0x28 + 0x166c = 0x16c0).

It consists of 3 parts:

1) timestamps

2) exposure values

3) index

The timestamp and exposure values parts start with a 4-byte length field allowing to skip them.

The index contains a 8-byte offset value for each frame. It does not have a length field, the number of entries is determined from the number of frames in the main header.

At each offset indicated by the index, there is first a header. The first 4 bytes of the header give the length of the header. Only the value 8 has a known meaning, in which case the next 4 bytes give the length of the frame data.

Phantom Cine

2010-03-28T11:43:36Z

Reimar:

Phantom Cine

2010-03-28T11:42:53Z

Reimar:

MPlayer youtube script

2009-12-30T20:54:12Z

Reimar: /* Or just use youtube-dl */

bash script for playing youtube videos

#save it as youtube.sh
#chmod +x youtube.sh
#place it somewhere in PATH (like /usr/local/bin )

Usage:
*youtube.sh <url> [mplayer args]
*youtube.sh http://www.youtube.com/watch?v=example -dumpstream -dumpfile something.flv
*youtube.sh http://www.youtube.com/watch?v=example -aspect 16:9
*youtube.sh http://www.youtube.com/watch?v=example -xy 2

<pre>
#!/bin/bash

if [ -z "$1" ]; then
echo "No URL!"
exit
fi

url=$1
shift

echo \"http://www.youtube.com/get_video?video_id=`wget -q -O - $url | grep fullscreenUrl | awk -F'video_id=' '{ print $2 }' | sed -e 's/ /_/g' | tr -d \'\; `\" | xargs mplayer $*

###
# The scipt above grabs the html source to the real stream (the .flv file), which youtube constantly alters. The "http://www.youtube.com/get_video?video_id=" is always the prefix -- and the script uses wget to d/l and append the rest of the 'full' url (which is very long and stupid).. an ex.;

#mplayer #"http://www.youtube.com/get_video?video_id=L2SED6sewRw&l=2965&sk=GOT2L_qmpVJOx0w#rud5ycdyuWziPg1lcC&fmt_map=6%2F720000%2F7%2F0%2F0&t=OEgsToPDskLlf4ls3xB6V84dMYLu#ndws&hl=en&plid=AARTXK76vCTnvQ5#JAAAC6ADCAAA&sdetail=rv%253AL2S#ED6sewRw&tk=P4Lg#O65y-u5BllZJBsB_e2Gw-OVgaMp4a8prsHTahDhuPN_xsReW2Q%3D%3D&title=Greg Kroah
# Hartman on the Linux Kernel"
# as you can see, there are blank spaces also that need to be cleaned up (blank # # spaces replaced with underscores).

</pre>

created by enouf and n3kl and amphi

RE-modified by enouf

===Alternative Version===
Here is an alternative version by [[User:Elte|Elte]].

The differences are
* fewer programm calls
* less shell pipes
* no dependency on grep, awk or tr
* supports double quotes in video url

This version is nevertheless heavily based on the previous script.
<pre>
#!/bin/bash
# © 2008,2009 Elte GPL V2.1

URL="$1"
shift

if [ "${URL#http://}" = "$URL" ]
then
echo "usage: $0 <youtube-URL> [mplayer args ...]"
exit
fi

echo \"http://www.youtube.com/get_video?video_id=$(wget -q -O - "$URL" \
| sed -e '/fullscreenUrl/!d' \
-e "s/.*video_id=$[^']*$.*/\1/" \
-e 's/ /_/g' \
-e 's/\\\"/"/g' \
)\" \
| xargs mplayer "$@"
</pre>

===Or just use youtube-dl===
Available in almost all Linux distributions.

Usage:

mplayer $(youtube-dl -g http://www.youtube.com/watch?v=.....)

or

mplayer $(youtube-dl -b -g http://www.youtube.com/watch?v=.....)

to get the HD version if available.

To avoid seeking issues, try updating to latest SVN. It contains a special hack that works around a bug in youtube's HTTP server (it does not set Accept-Ranges, which incorrectly indicates it does not support seeking/partial requests - MPlayer now just assumes that seeking is possible if the server string is "gvs 1.0" - which is what youtube currently uses).

If that does not help, try configuring MPlayer with http support via FFmpeg (--enable-protocol="http_protocol" or --enable-protocol="file_protocol pipe_protocol http_protocol rtmp_protocol tcp_protocol udp_protocol") and use instead

mplayer ffmpeg://$(youtube-dl -b -g http://www.youtube.com/watch?v=.....)

MPlayer youtube script

2009-12-30T20:30:46Z

Reimar: /* Or just use youtube-dl */

bash script for playing youtube videos

#save it as youtube.sh
#chmod +x youtube.sh
#place it somewhere in PATH (like /usr/local/bin )

Usage:
*youtube.sh <url> [mplayer args]
*youtube.sh http://www.youtube.com/watch?v=example -dumpstream -dumpfile something.flv
*youtube.sh http://www.youtube.com/watch?v=example -aspect 16:9
*youtube.sh http://www.youtube.com/watch?v=example -xy 2

<pre>
#!/bin/bash

if [ -z "$1" ]; then
echo "No URL!"
exit
fi

url=$1
shift

echo \"http://www.youtube.com/get_video?video_id=`wget -q -O - $url | grep fullscreenUrl | awk -F'video_id=' '{ print $2 }' | sed -e 's/ /_/g' | tr -d \'\; `\" | xargs mplayer $*

###
# The scipt above grabs the html source to the real stream (the .flv file), which youtube constantly alters. The "http://www.youtube.com/get_video?video_id=" is always the prefix -- and the script uses wget to d/l and append the rest of the 'full' url (which is very long and stupid).. an ex.;

#mplayer #"http://www.youtube.com/get_video?video_id=L2SED6sewRw&l=2965&sk=GOT2L_qmpVJOx0w#rud5ycdyuWziPg1lcC&fmt_map=6%2F720000%2F7%2F0%2F0&t=OEgsToPDskLlf4ls3xB6V84dMYLu#ndws&hl=en&plid=AARTXK76vCTnvQ5#JAAAC6ADCAAA&sdetail=rv%253AL2S#ED6sewRw&tk=P4Lg#O65y-u5BllZJBsB_e2Gw-OVgaMp4a8prsHTahDhuPN_xsReW2Q%3D%3D&title=Greg Kroah
# Hartman on the Linux Kernel"
# as you can see, there are blank spaces also that need to be cleaned up (blank # # spaces replaced with underscores).

</pre>

created by enouf and n3kl and amphi

RE-modified by enouf

===Alternative Version===
Here is an alternative version by [[User:Elte|Elte]].

The differences are
* fewer programm calls
* less shell pipes
* no dependency on grep, awk or tr
* supports double quotes in video url

This version is nevertheless heavily based on the previous script.
<pre>
#!/bin/bash
# © 2008,2009 Elte GPL V2.1

URL="$1"
shift

if [ "${URL#http://}" = "$URL" ]
then
echo "usage: $0 <youtube-URL> [mplayer args ...]"
exit
fi

echo \"http://www.youtube.com/get_video?video_id=$(wget -q -O - "$URL" \
| sed -e '/fullscreenUrl/!d' \
-e "s/.*video_id=$[^']*$.*/\1/" \
-e 's/ /_/g' \
-e 's/\\\"/"/g' \
)\" \
| xargs mplayer "$@"
</pre>

===Or just use youtube-dl===
Available in almost all Linux distributions.

Usage:

mplayer $(youtube-dl -g http://www.youtube.com/watch?v=.....)

or

mplayer $(youtube-dl -b -g http://www.youtube.com/watch?v=.....)

to get the HD version if available.

To resolve seeking issues, try configuring MPlayer with http support via FFmpeg (--enable-protocol="http_protocol" or --enable-protocol="file_protocol pipe_protocol http_protocol rtmp_protocol tcp_protocol udp_protocol") and use instead

mplayer ffmpeg://$(youtube-dl -b -g http://www.youtube.com/watch?v=.....)

MPlayer youtube script

2009-12-30T20:25:24Z

Reimar: /* Or just use youtube-dl */

MPlayer youtube script

2009-12-30T20:21:54Z

Reimar: Or just use youtube-dl...

bash script for playing youtube videos

#save it as youtube.sh
#chmod +x youtube.sh
#place it somewhere in PATH (like /usr/local/bin )

Usage:
*youtube.sh <url> [mplayer args]
*youtube.sh http://www.youtube.com/watch?v=example -dumpstream -dumpfile something.flv
*youtube.sh http://www.youtube.com/watch?v=example -aspect 16:9
*youtube.sh http://www.youtube.com/watch?v=example -xy 2

<pre>
#!/bin/bash

if [ -z "$1" ]; then
echo "No URL!"
exit
fi

url=$1
shift

echo \"http://www.youtube.com/get_video?video_id=`wget -q -O - $url | grep fullscreenUrl | awk -F'video_id=' '{ print $2 }' | sed -e 's/ /_/g' | tr -d \'\; `\" | xargs mplayer $*

###
# The scipt above grabs the html source to the real stream (the .flv file), which youtube constantly alters. The "http://www.youtube.com/get_video?video_id=" is always the prefix -- and the script uses wget to d/l and append the rest of the 'full' url (which is very long and stupid).. an ex.;

#mplayer #"http://www.youtube.com/get_video?video_id=L2SED6sewRw&l=2965&sk=GOT2L_qmpVJOx0w#rud5ycdyuWziPg1lcC&fmt_map=6%2F720000%2F7%2F0%2F0&t=OEgsToPDskLlf4ls3xB6V84dMYLu#ndws&hl=en&plid=AARTXK76vCTnvQ5#JAAAC6ADCAAA&sdetail=rv%253AL2S#ED6sewRw&tk=P4Lg#O65y-u5BllZJBsB_e2Gw-OVgaMp4a8prsHTahDhuPN_xsReW2Q%3D%3D&title=Greg Kroah
# Hartman on the Linux Kernel"
# as you can see, there are blank spaces also that need to be cleaned up (blank # # spaces replaced with underscores).

</pre>

created by enouf and n3kl and amphi

RE-modified by enouf

===Alternative Version===
Here is an alternative version by [[User:Elte|Elte]].

The differences are
* fewer programm calls
* less shell pipes
* no dependency on grep, awk or tr
* supports double quotes in video url

This version is nevertheless heavily based on the previous script.
<pre>
#!/bin/bash
# © 2008,2009 Elte GPL V2.1

URL="$1"
shift

if [ "${URL#http://}" = "$URL" ]
then
echo "usage: $0 <youtube-URL> [mplayer args ...]"
exit
fi

echo \"http://www.youtube.com/get_video?video_id=$(wget -q -O - "$URL" \
| sed -e '/fullscreenUrl/!d' \
-e "s/.*video_id=$[^']*$.*/\1/" \
-e 's/ /_/g' \
-e 's/\\\"/"/g' \
)\" \
| xargs mplayer "$@"
</pre>

===Or just use youtube-dl===
Available in almost all Linux distributions.
Usage:
mplayer $(youtube-dl -g http://www.youtube.com/watch?v=.....)

IMA ADPCM

2009-06-24T13:23:42Z

Reimar: /* Optimization */

The [[Interactive Multimedia Association]] (IMA) developed an ADPCM algorithm designed to be used in entertainment multimedia applications. It is particularly fast to encode and decode and does not strictly require any multiplications or floating point operations.

While the encoding and decoding algorithms remain more or less constant across different IMA implementations, the specific on-disk data formats vary. This page describes the common IMA decoding algorithm. See the [[:Category:IMA ADPCM Audio Codecs]] page for various formats used for storing the data on disk.

== Decoding IMA ==
To decode IMA ADPCM, initialize 3 variables:

* predictor: This is either initialized from the data chunk preamble specified in the format or is initialized to 0 at the start of the decoding process.
* step index: Similar to the initial predictor, this variable is initialized from the data chunk preamble or set to 0 at the start of the decoding process.
* step: This variable is initialized to ima_step_table[step_index].

The encoded IMA bitstream is comprised of a series of 4-bit nibbles. This means that each byte represents 2 IMA nibbles. The specific data format will dictate whether the stream is decoded top nibble first or bottom nibble first, and whether there is stereo interleaving within the IMA nibbles. For this discussion, imagine the IMA bitstream as a series of nibbles representing a single audio channel:

n0 n1 n2 n3 n4 n5 ...

Where each nibble represents both a table index and a sign/magnitude number during the decoding process. Transform each nibble in the stream into a signed, 16-bit PCM sample using the following process:

step_index = step_index + ima_index_table[(unsigned)nibble]

diff = ((signed)nibble + 0.5) * step / 4

predictor = predictor + diff

step = ima_step_table[step index]

Regarding the step index and predictor calculations: Be sure to saturate the computed step index between 0 and 88 (table limits) and the predictor between -32768 and 32767 (signed 16-bit number range). It is possible for these values to outrange which could cause undesirable program behavior if unchecked.

== Optimization ==

A note about the following calculation:

diff = ((sign/mag.)nibble + 0.5) * step / 4

At first glance, it appears that this calculation requires floating point operations and an arbitrary (not power-of-2) multiplication. However, some numerical manipulations reveal some useful simplifications:

diff = ((step * nibble) + (step / 2)) / 4

diff = (step * nibble / 4) + (step / 8)

The step / 8 calculation can be expressed as a bit shift right by 3 (step SHR 3). The first part of the equation can also be simplified. Since a nibble only carries 4 bits, and those 4 bits are a sign/magnitude number, there are only 3 bits of magnitude information. If all 3 magnitude bits are set to 1:

nibble = 4 + 2 + 1

step * nibble / 4 = (4 * step / 4) + (2 * step / 4) + (1 * step / 4) = step + (step / 2) + (step / 4)

Thus, if bit 2 of the nibble is set, add step to diff. If bit 1 is set, add (step / 2 = step SHR 1) to diff. If bit 0 is set, add (step / 4 = step SHR 2) to diff. Finally, if the sign bit is set, subtract the final diff value from the predictor value; otherwise, add the final diff value to the predictor value. The usual algorithm is as follows:

sign = nibble & 8
delta = nibble & 7
diff = step >> 3
if (delta & 4) diff += step
if (delta & 2) diff += (step >> 1)
if (delta & 1) diff += (step >> 2)
if (sign) predictor -= diff
else predictor += diff

This method was particularly useful back when IMA was implemented on commodity CPUs which were relatively slow at multiplication. One multiplication per audio sample had a notable impact on program performance, as opposed to the series of branches, additions and logical bit operations. If multiplication performance is not an issue, it is possible to carry out the diff calculation with only one non-power-of-2 multiplication and no floating point numbers:

diff = ((((signed)nibble+0.5) * step) / 4) * (2 / 2)

diff = (nibble + 0.5) * 2 * step / 8

diff = (2 * nibble + 1) * step / 8

[NOTE: something seems wrong here, e.g. for nibble = 3 and step = 3 we would get with the upper formula
diff = 0 + 1 + 0 = 1
and with the lower one
diff = (2 * 3 + 1) * 3 / 8 = 21 / 8 = 2
Since with ADPCM errors can propagate forever that seems like a really bad thing...
]

== Decoding Tables ==

int index_table[16] = {
-1, -1, -1, -1, 2, 4, 6, 8,
-1, -1, -1, -1, 2, 4, 6, 8
};

Note that many programs use slight deviations from the following table, but such deviations are negligible:

int step_table[89] = {
7, 8, 9, 10, 11, 12, 13, 14, 16, 17,
19, 21, 23, 25, 28, 31, 34, 37, 41, 45,
50, 55, 60, 66, 73, 80, 88, 97, 107, 118,
130, 143, 157, 173, 190, 209, 230, 253, 279, 307,
337, 371, 408, 449, 494, 544, 598, 658, 724, 796,
876, 963, 1060, 1166, 1282, 1411, 1552, 1707, 1878, 2066,
2272, 2499, 2749, 3024, 3327, 3660, 4026, 4428, 4871, 5358,
5894, 6484, 7132, 7845, 8630, 9493, 10442, 11487, 12635, 13899,
15289, 16818, 18500, 20350, 22385, 24623, 27086, 29794, 32767
};

[[Category:Audio Codecs]]
[[Category:ADPCM Audio Codecs]]
[[Category:IMA ADPCM Audio Codecs]]

UMV

2009-05-15T10:51:24Z

Reimar:

* Extension: UMV
* Samples: http://samples.mplayerhq.hu/game-formats/umv/

UMV is (suspected to be) a full motion video file format used in the DOS game [http://www.mobygames.com/game/dos/are-you-afraid-of-the-dark-the-tale-of-orpheos-curse Are You Afraid of the Dark? The Tale of Orpheo's Curse].

[[Category:Video Codecs]]
[[Category:Game Formats]]
[[Category:Undiscovered Game Formats]]

The format is split into individual "packets", with no special header.
Each packet starts with 4 bytes packet size (big-endian, including the 4 bytes for the size itself), followed by the size value of the previous packet (0 if there is no previous one).
The following 4 bytes are probably some kind of "packet type" value.
Packets are grouped together, with the last packet in a group having a size value of 0 and type 00 00 00 80.
After each packet group follows a section of PCM audio samples, up to the start of the next packet group.
The start offset of the next packet group is coded in the first packet of the current group, such a packet starts e.g. like this:
00 00 00 40 00 00 00 00 00 00 00 02 00 02 c8 00
where 0x40 is the size of that first packet, 0 is the size of the previous one, the packet "type" is 2 and 0x2c800 is the start offset of the next packet group.

The PCM "packets" are strange, e.g. in V_END7.UMV most packets seem to have only a size of 8820 bytes, the remaining bytes are filled up with the value 0x80 - whereas the e.g. first PCM packet and the one starting at 0x47f2c seem to contain audio data all the way.
Possibly the value 0x80 is not allowed in the signed PCM format used but instead indicates padding/values to be dropped?

Video seems to be a paletted format, where the palette is stored in packets of type 0x40 in a 1-byte per component, 3 component format (i.e. 768 bytes for a full palette). It is possible that these palette packets specify the first and last palette entry they replace, so only parts of the palette can be changed.
Packets of type 0x20 might code full frames/keyframes at a resolution of 160x120.

UMV

2009-05-15T10:40:27Z

Reimar:

UMV

2009-05-15T10:25:00Z

Reimar:

UMV

2009-05-14T21:13:57Z

Reimar:

UMV

2009-05-14T21:12:26Z

Reimar:

CDXL

2009-04-08T14:09:30Z

Reimar: /* Video */

* Company: [[Commodore]]
* Patents: US 5,293,606, "Apparatus and method for transferring interleaved data objects in mass storage devices into separate destinations in memory"
* See also: [http://en.wikipedia.org/wiki/CDXL Wikipedia CDXL article], [http://web.archive.org/web/20030108081402/http://home.t-online.de/home/K_Andreas/CDXL.HTM German article with technical details] (archive.org)
* Amiga decoder with source code in Pascal: http://aminet.net/package/gfx/show/AnimFX
* Sample: ftp://ffmpeg.org/MPlayer/samples/game-formats/cdxl/ , http://valerio.diinoweb.com/files/amigaanimecd/index.htm

CDXL is an uncompressed video format created by Commodore for playback from CD-ROM on Amiga computers.

== File format ==

CDXL files have no identifying markers or file headers; the file contains a number of frames, each prefixed by a frame header.

Frame header | Palette | Video | Audio

All multi-byte integers are big endian.

== Frame header ==

byte 0 File type
byte 1 Info byte
bits 0-2 Video encoding
bit 3 Stereo flag
bits 5-7 Plane arrangement
bytes 2-5 Current chunk size
bytes 6-9 Previous chunk size
bytes 10-11 Reserved
bytes 12-13 Current frame number (1 for first frame)
bytes 14-15 Video width
bytes 16-17 Video height
bytes 18-19 Number of bit planes
bytes 20-21 Palette size in bytes
bytes 22-23 Sound size in bytes
bytes 24-31 Reserved

File type values (it is unknown what these mean):

0 Custom CDXL
1 Standard CDXL
2 Special CDXL

Video encoding values:

0 RGB
1 HAM
2 YUV
3 AVM & DCTV

Plane arrangement values:
0 Bit planar
1 Byte planar
2 Chunky
4 Bit line
6 Byte line

== Palette ==

The palette is encoded as 12-bit RGB values (4 bits each of R, G, and B) stored in 16-bit words with the upper 4 bits unused and set to 0. The palette size field in the header is the number of bytes in the palette, i.e. double the number of entries in the palette.

== Audio ==

Audio is encoded in standard uncompressed signed 8-bit PCM. The CDXL file itself does not seem to contain any sampling rate information, although related documents suggest 11025 Hz is standard.

== Video ==

Video is encoded differently based on the info byte in the header.

RGB is encoded as an index into the palette for the current frame.

HAM (Hold-And-Modify), an Amiga-specific video mode, is encoded as described on [http://en.wikipedia.org/wiki/Hold-And-Modify Wikipedia].
The sample at http://samples.mplayerhq.hu/game-formats/cdxl/amigaball.cdxl at least seems to use the HAM6 method, since it has 32 bytes
of palette (i.e. 16 palette entries).

[[Category:Container Formats]]
[[Category:Video Codecs]]
[[Category:Incomplete Video Codecs]]

FFmpeg Summer Of Code 2009

2009-03-20T09:51:44Z

Reimar: /* AACS implementation */

== Current Status ==

This list is still a work-in-progress, please see also the [[Talk:FFmpeg Summer Of Code 2009|Talk Page]].

== Qualification tasks ==

For us to consider your application for SoC we require a completed qualification task. Choose a task from the [[Small FFmpeg Tasks|Small Tasks list]], send an email to FFmpeg-devel mailing list to inform that you are working on it (to avoid duplicated work) and when it is ready submit it for review at FFmpeg-devel. The task is considered completed when your patch is accepted to our main SVN tree.

== 1st Tier Project Proposals ==
1st tier project proposals are project ideas that are reasonably well defined '''AND''' have a mentor volunteered.

=== S/PDIF muxer ===
* Implement a muxer capable to mux:
** DTS, all 3 packing modes and the usable HD extensions
** AC3, eAC3 also
** MLP
** PCM
** WMApro
** AAC
** Mpeg-audio, layer 2 and 3

Implement support in ffplay so that it is possible to output the audio stream over S/PDIF when playing a media file.

''Mentor: Benjamin Larsson''

=== Flash Screen video 2 codec ===
* Implement a flashsv2 decoder and encoder. And extend the current flashsv encoder to support optimal 2-pass encoding.
''Mentor: Benjamin Larsson''

=== MPEG-4 ALS decoder ===
*primary goal: stream copy of ALS frames in MP4 files from reference encoder
** detect codec_id
** preserve extradata
*primary goal: write the decoder based on the ISO specification
** ISO/IEC 14496-3:2005/Amd.2:2006 and related corrigenda
*primary goal: decode files with basic ALS features
** integer samples
** LPC
** rice coding
** joint-stereo
*secondary goal: decode files with more advanced ALS features
** floating-point samples
** block switching
** LTP (long term prediction)
** BGMC (arithmetic coding)
** MCC (advanced multi-channel)
** RLSLMS (backward-adaptive prediction)
*secondary goal: pass the ISO conformance tests
*secondary goal: handle anything the reference encoder can come up with
''Mentor: Justin Ruggles''

=== Playlist/Concatenation Support ===
*primary goal: implement a playlist/concatenation interface
to transcode(FFmpeg) and play(FFplay) media
** interface will use commandline switches.
** interface must support every input format FFmpeg support
** interface must work with different input stream parameters (different formats, codecs, video resolution, audio sample rate, audio channels, etc..)
** interface must support track selection
** interface must support existing playlist format files .m3u, .pls, xpsf.

''Mentor: Baptiste Coudurier''

=== NEW Seeking API ===
*primary goal: implement a new seeking API in libavformat
** implement av_seek_file in libavformat
** implement compatible new seek_file for all AVInputFormat porting existing seek function if possible.
** implement av_build_index function which will build an AVIndex for the file
** implement av_export_index function which will save AVIndex in a file which can be loaded later.

''Mentor: Baptiste Coudurier''

=== Improve RTSP/RTP layer ===
*primary goal: improve the receiver compatibility
** Add support for more widespread formats ([list will follow check gst live555 and feng])
*** X-Qt/quicktime depayloader (see [http://www.gnome.org/~rbultje/ffmpeg-patchset/ X-QT patch])
*** vorbis and theora depayloader (see [[Small_FFmpeg_Tasks#Implement_the_RTP.2FVorbis_payload]])
*** h263 and h263+ (see [http://roundup.ffmpeg.org/roundup/ffmpeg/issue678 Issue 678])
*** ...more...
** support Quicktime http tunnel mode
*secondary goal: provide an API to expose the rtcp layer (and the equivalent in RDT dialect)
*secondary goal: try to support subtitle streams (either as rtcp-xr or application/text stream)
*secondary goal: make VideoLanClient, MPlayer and Xine use ffmpeg rtsp

''Mentor: Luca Barbato, Ronald S. Bultje''

=== AACS implementation ===
* Add the ability to encode and decode using Advanced Access Content System to FFmpeg.
* Specifications: http://www.aacsla.com/specifications/
* existing implementation e.g. DumpHD: http://forum.doom9.org/showthread.php?t=123111
* Most parts (BD-J, MKB, title key generation) probably do not belong into FFmpeg, this should be discussed with us before submitting an application
** possible solution: only implement "lowest" level (decode given the correct title key) but implement CSS en- and decryption as secondary goal

''Mentor: Reimar Döffinger''

== 2nd Tier Project Proposals ==
All that separates these proposals from their 1st tier brethren is a mentor.

=== Finish SoC projects from previous years ===
Some projects are lingering in the dark unfinished. They should be picked up and made ready for inclusion. These projects are potentially less involved than starting from scratch, but also more useful for FFmpeg since the probability that the projects get finished should be higher. If some of them are deemed too easy, they could be combined.

Unfinished projects from previous years are:

2006:
* AMR-NB decoder

2007:
* QCELP decoder (missing features)
* JPEG 2000 decoder
* JPEG 2000 encoder
* Dirac decoder
* Dirac encoder
* TS muxer

2008:
* Generic frame-level multithreading support
* AAC-LC encoder
* MLP/TrueHD encoder
* WMA Pro decoder

=== [[Libavfilter]] video work ===
Libavfilter is the FFmpeg filtering library that started as a 2007 SoC [[FFmpeg Summer Of Code#Video Filter API (AKA libavfilter)|project]]. It should replace the now removed vhook subsystem. Most of it is already part of the FFmpeg main source tree, but there a few bits remaining. This project would consist in the following tasks

* Get the remaining bits of the SoC tree committed, including the ffmpeg.c and ffplay.c patch
* Get libavfilter enabled in the main SVN tree
* Write a watermark filter (this is one of the most commonly requested FFmpeg feature)
* Write a expand/pad filter (see [http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/85015] and [http://thread.gmane.org/gmane.comp.video.ffmpeg.soc/2779/]
* Port all MPlayer filters at libmbcodec/vf_* (do not forget asking the authors if it is ok to release them under the LGPL)

see also this ffmpeg-devel message: [http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-March/064817.html]

=== [[Libavfilter]] audio work ===
At the moment, FFmpeg filtering library has no support at all for handling audio. This task would consist of

* Expanding the libavfilter framework to work with audio
* Writing a resampling filter (starting with just a wrapping code at libavcodec/audioconvert.c)
* Implement negotiation of sample format and number of channels analogously to the libavfilter colorspace negotiation
* Make the resampling filter works for several combinations of sample format and channels
* Write a visualization filter as proof-of-concept of a filter that works with both video and audio

=== Implement a better regressions test system ===
* Split up the current regtests
* Add tests for all the missing formats and codecs to FATE

=== libvo ===
* Port MPlayer's libvo to ffplay
* Note that this does not just mean to produce a working hack so that ffplay can use xv, but a clean and acceptable wrapper for (most of) libvo.

=== GStreamer input ===
* Like we have vfw input we could have a gstreamer input format also. This would enable support of wmapro and wmalossless until these formats are RE'd.
''Mentor: Christian Schaller''

=== AMR-WB Decoder ===
* Specification: http://www.3gpp.org/ftp/Specs/html-info/26-series.htm
* Samples: http://samples.mplayerhq.hu/A-codecs/amr/
Also see [[AMR]].

=== GSM Decoder ===
* Specification + sample implementation: http://kbs.cs.tu-berlin.de/~jutta/toast.html
* Samples: http://samples.mplayerhq.hu/A-codecs/GSM/
Also see [[GSM]].

=== Sipr Decoder ===
* Specification: will be provided
* Samples: http://samples.mplayerhq.hu/real/AC-sipr/
Also see [[RealAudio sipr]] and [[Interesting_Patches#RealAudio_SIPR_.4016k_decoder_and_demuxer_by_Vladimir_Voroshilov|this patch]].

=== Speex Decoder ===
* Specification: http://speex.org/docs/
* Samples: http://samples.mplayerhq.hu/A-codecs/speex/
Also see [[Speex]].

=== AMR-NB Encoder ===
* Specification: http://www.3gpp.org/ftp/Specs/html-info/26-series.htm
* Samples: http://samples.mplayerhq.hu/A-codecs/amr/
Also see [[AMR]].

=== VP6 Encoder ===
* Specification: [[On2 VP6]]
* Samples: http://samples.mplayerhq.hu/V-codecs/VP6/

=== WMV3 Encoder ===
* Clearly defined task
* Primary goal: Encode video sequences such that they can be decoded by a Windows Media player.

This could either be done by improving [[Interesting Patches#WMV3 encoder by Denis Fortin|this patch]] or by writing the encoder from scratch.

=== Improve subtitle support ===

* Add text-to-bitmap conversion functions
* One with hard-coded bitmaps for characters
* One that utilizes freetype
* Function used will be chosen upon compilation

Adjust existing subtitle support to new ABI
* ABI change: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-January/058521.html

=== VC-1 Interlaced Support ===
* Add support for interlaced streams as used in Bluray recordings to the VC-1 decoder.
* This includes fixing some reference streams

=== Improve Ratecontrol ===
*Primary goal 1: Fast heuristic VBV compliant per macroblock ratecontrol which has a better PSNR/bitrate and better subjective quality/bitrate than the current code.
*Primary goal 2: VBV compliant, rate distortion optimal per macroblock ratecontrol using the viterbi algorithm.
*Secondary goal 1: Fast heuristic scene change detection which detects scene changes more accurately, has better PSNR/bitrate and subjective quality/bitrate than the current heuristic.
*Secondary goal 2: Rate distortion optimal (for the current picture) scene change detection.
*Secondary goal 3: B frames decision which is faster and or has a higher PSNR/bitrate and subjective quality/bitrate than the current code.

=== WMA lossless ===
* Implement a decoder for WMA lossless (0x0163)
* Reuse as much libavcodec code as possible
* Samples: http://samples.mplayerhq.hu/A-codecs/WMA9/wma_0x163.wma http://samples.mplayerhq.hu/A-codecs/lossless/luckynight.wma

=== WTV (de)muxer ===
* Implement a demuxer (and possibly a muxer) for the [[WTV]] file format.

FFmpeg Summer Of Code 2009

2009-03-19T18:52:37Z

Reimar: DumpHD AACS implementation

== Current Status ==

This list is still a work-in-progress, please see also the [[Talk:FFmpeg Summer Of Code 2009|Talk Page]].

== Qualification tasks ==

For us to consider your application for SoC we require a completed qualification task. Choose a task from the [[Small FFmpeg Tasks|Small Tasks list]], send an email to FFmpeg-devel mailing list to inform that you are working on it (to avoid duplicated work) and when it is ready submit it for review at FFmpeg-devel. The task is considered completed when your patch is accepted to our main SVN tree.

== 1st Tier Project Proposals ==
1st tier project proposals are project ideas that are reasonably well defined '''AND''' have a mentor volunteered.

=== S/PDIF muxer ===
* Implement a muxer capable to mux:
** DTS, all 3 packing modes and the usable HD extensions
** AC3, eAC3 also
** MLP
** PCM
** WMApro
** AAC
** Mpeg-audio, layer 2 and 3

Implement support in ffplay so that it is possible to output the audio stream over S/PDIF when playing a media file.

''Mentor: Benjamin Larsson''

=== Flash Screen video 2 codec ===
* Implement a flashsv2 decoder and encoder. And extend the current flashsv encoder to support optimal 2-pass encoding.
''Mentor: Benjamin Larsson''

=== MPEG-4 ALS decoder ===
*primary goal: stream copy of ALS frames in MP4 files from reference encoder
** detect codec_id
** preserve extradata
*primary goal: write the decoder based on the ISO specification
** ISO/IEC 14496-3:2005/Amd.2:2006 and related corrigenda
*primary goal: decode files with basic ALS features
** integer samples
** LPC
** rice coding
** joint-stereo
*secondary goal: decode files with more advanced ALS features
** floating-point samples
** block switching
** LTP (long term prediction)
** BGMC (arithmetic coding)
** MCC (advanced multi-channel)
** RLSLMS (backward-adaptive prediction)
*secondary goal: pass the ISO conformance tests
*secondary goal: handle anything the reference encoder can come up with
''Mentor: Justin Ruggles''

=== Playlist/Concatenation Support ===
*primary goal: implement a playlist/concatenation interface
to transcode(FFmpeg) and play(FFplay) media
** interface will use commandline switches.
** interface must support every input format FFmpeg support
** interface must work with different input stream parameters (different formats, codecs, video resolution, audio sample rate, audio channels, etc..)
** interface must support track selection
** interface must support existing playlist format files .m3u, .pls, xpsf.

''Mentor: Baptiste Coudurier''

=== NEW Seeking API ===
*primary goal: implement a new seeking API in libavformat
** implement av_seek_file in libavformat
** implement compatible new seek_file for all AVInputFormat porting existing seek function if possible.
** implement av_build_index function which will build an AVIndex for the file
** implement av_export_index function which will save AVIndex in a file which can be loaded later.

''Mentor: Baptiste Coudurier''

=== Improve RTSP/RTP layer ===
*primary goal: improve the receiver compatibility
** Add support for more widespread formats ([list will follow check gst live555 and feng])
*** X-Qt/quicktime depayloader (see [http://www.gnome.org/~rbultje/ffmpeg-patchset/ X-QT patch])
*** vorbis and theora depayloader (see [[Small_FFmpeg_Tasks#Implement_the_RTP.2FVorbis_payload]])
*** h263 and h263+ (see [http://roundup.ffmpeg.org/roundup/ffmpeg/issue678 Issue 678])
*** ...more...
** support Quicktime http tunnel mode
*secondary goal: provide an API to expose the rtcp layer (and the equivalent in RDT dialect)
*secondary goal: try to support subtitle streams (either as rtcp-xr or application/text stream)
*secondary goal: make VideoLanClient, MPlayer and Xine use ffmpeg rtsp

''Mentor: Luca Barbato, Ronald S. Bultje''

== 2nd Tier Project Proposals ==
All that separates these proposals from their 1st tier brethren is a mentor.

=== [[Libavfilter]] video work ===
Libavfilter is the FFmpeg filtering library that started as a 2007 SoC [[FFmpeg Summer Of Code#Video Filter API (AKA libavfilter)|project]]. It should replace the now removed vhook subsystem. Most of it is already part of the FFmpeg main source tree, but there a few bits remaining. This project would consist in the following tasks

* Get the remaining bits of the SoC tree committed, including the ffmpeg.c and ffplay.c patch
* Get libavfilter enabled in the main SVN tree
* Write a watermark filter (this is one of the most commonly requested FFmpeg feature)
* Write a expand/pad filter (see [http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/85015] and [http://thread.gmane.org/gmane.comp.video.ffmpeg.soc/2779/]
* Port all MPlayer filters at libmbcodec/vf_* (do not forget asking the authors if it is ok to release them under the LGPL)

see also this ffmpeg-devel message: [http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-March/064817.html]

=== [[Libavfilter]] audio work ===
At the moment, FFmpeg filtering library has no support at all for handling audio. This task would consist of

* Expanding the libavfilter framework to work with audio
* Writing a resampling filter (starting with just a wrapping code at libavcodec/audioconvert.c)
* Implement negotiation of sample format and number of channels analogously to the libavfilter colorspace negotiation
* Make the resampling filter works for several combinations of sample format and channels
* Write a visualization filter as proof-of-concept of a filter that works with both video and audio

=== Implement a better regressions test system ===
* Split up the current regtests
* Add tests for all the missing formats and codecs to FATE

=== libvo ===
* Port MPlayer's libvo to ffplay
* Note that this does not just mean to produce a working hack so that ffplay can use xv, but a clean and acceptable wrapper for (most of) libvo.

=== GStreamer input ===
* Like we have vfw input we could have a gstreamer input format also. This would enable support of wmapro and wmalossless until these formats are RE'd.
''Mentor: Christian Schaller''

=== AMR-WB Decoder ===
* Specification: http://www.3gpp.org/ftp/Specs/html-info/26-series.htm
* Samples: http://samples.mplayerhq.hu/A-codecs/amr/
Also see [[AMR]].

=== GSM Decoder ===
* Specification + sample implementation: http://kbs.cs.tu-berlin.de/~jutta/toast.html
* Samples: http://samples.mplayerhq.hu/A-codecs/GSM/
Also see [[GSM]].

=== Sipr Decoder ===
* Specification: will be provided
* Samples: http://samples.mplayerhq.hu/real/AC-sipr/
Also see [[RealAudio sipr]] and [[Interesting_Patches#RealAudio_SIPR_.4016k_decoder_and_demuxer_by_Vladimir_Voroshilov|this patch]].

=== Speex Decoder ===
* Specification: http://speex.org/docs/
* Samples: http://samples.mplayerhq.hu/A-codecs/speex/
Also see [[Speex]].

=== AMR-NB Encoder ===
* Specification: http://www.3gpp.org/ftp/Specs/html-info/26-series.htm
* Samples: http://samples.mplayerhq.hu/A-codecs/amr/
Also see [[AMR]].

=== VP6 Encoder ===
* Specification: [[On2 VP6]]
* Samples: http://samples.mplayerhq.hu/V-codecs/VP6/

=== WMV3 Encoder ===
* Clearly defined task
* Primary goal: Encode video sequences such that they can be decoded by a Windows Media player.

This could either be done by improving [[Interesting Patches#WMV3 encoder by Denis Fortin|this patch]] or by writing the encoder from scratch.

=== Improve subtitle support ===

* Add text-to-bitmap conversion functions
* One with hard-coded bitmaps for characters
* One that utilizes freetype
* Function used will be chosen upon compilation

Adjust existing subtitle support to new ABI
* ABI change: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-January/058521.html

=== AACS implementation ===
* Add the ability to encode and decode using Advanced Access Content System to FFmpeg.
* Specifications: http://www.aacsla.com/specifications/
* existing implementation e.g. DumpHD: http://forum.doom9.org/showthread.php?t=123111
* Most parts (BD-J, MKB, title key generation) probably do not belong into FFmpeg, this should be discussed with us before submitting an application

''Mentor: Reimar Döffinger''

=== VC-1 Interlaced Support ===
* Add support for interlaced streams as used in Bluray recordings to the VC-1 decoder.
* This includes fixing some reference streams

=== Improve Ratecontrol ===
*Primary goal 1: Fast heuristic VBV compliant per macroblock ratecontrol which has a better PSNR/bitrate and better subjective quality/bitrate than the current code.
*Primary goal 2: VBV compliant, rate distortion optimal per macroblock ratecontrol using the viterbi algorithm.
*Secondary goal 1: Fast heuristic scene change detection which detects scene changes more accurately, has better PSNR/bitrate and subjective quality/bitrate than the current heuristic.
*Secondary goal 2: Rate distortion optimal (for the current picture) scene change detection.
*Secondary goal 3: B frames decision which is faster and or has a higher PSNR/bitrate and subjective quality/bitrate than the current code.

=== WMA lossless ===
* Implement a decoder for WMA lossless (0x0163)
* Reuse as much libavcodec code as possible
* Samples: http://samples.mplayerhq.hu/A-codecs/WMA9/wma_0x163.wma http://samples.mplayerhq.hu/A-codecs/lossless/luckynight.wma

=== WTV (de)muxer ===
* Implement a demuxer (and possibly a muxer) for the [[WTV]] file format.

FFmpeg Summer Of Code 2009

2009-03-19T18:50:41Z

Reimar: /* AACS implementation */

== Current Status ==

This list is still a work-in-progress, please see also the [[Talk:FFmpeg Summer Of Code 2009|Talk Page]].

== Qualification tasks ==

For us to consider your application for SoC we require a completed qualification task. Choose a task from the [[Small FFmpeg Tasks|Small Tasks list]], send an email to FFmpeg-devel mailing list to inform that you are working on it (to avoid duplicated work) and when it is ready submit it for review at FFmpeg-devel. The task is considered completed when your patch is accepted to our main SVN tree.

== 1st Tier Project Proposals ==
1st tier project proposals are project ideas that are reasonably well defined '''AND''' have a mentor volunteered.

=== S/PDIF muxer ===
* Implement a muxer capable to mux:
** DTS, all 3 packing modes and the usable HD extensions
** AC3, eAC3 also
** MLP
** PCM
** WMApro
** AAC
** Mpeg-audio, layer 2 and 3

Implement support in ffplay so that it is possible to output the audio stream over S/PDIF when playing a media file.

''Mentor: Benjamin Larsson''

=== Flash Screen video 2 codec ===
* Implement a flashsv2 decoder and encoder. And extend the current flashsv encoder to support optimal 2-pass encoding.
''Mentor: Benjamin Larsson''

=== MPEG-4 ALS decoder ===
*primary goal: stream copy of ALS frames in MP4 files from reference encoder
** detect codec_id
** preserve extradata
*primary goal: write the decoder based on the ISO specification
** ISO/IEC 14496-3:2005/Amd.2:2006 and related corrigenda
*primary goal: decode files with basic ALS features
** integer samples
** LPC
** rice coding
** joint-stereo
*secondary goal: decode files with more advanced ALS features
** floating-point samples
** block switching
** LTP (long term prediction)
** BGMC (arithmetic coding)
** MCC (advanced multi-channel)
** RLSLMS (backward-adaptive prediction)
*secondary goal: pass the ISO conformance tests
*secondary goal: handle anything the reference encoder can come up with
''Mentor: Justin Ruggles''

=== Playlist/Concatenation Support ===
*primary goal: implement a playlist/concatenation interface
to transcode(FFmpeg) and play(FFplay) media
** interface will use commandline switches.
** interface must support every input format FFmpeg support
** interface must work with different input stream parameters (different formats, codecs, video resolution, audio sample rate, audio channels, etc..)
** interface must support track selection
** interface must support existing playlist format files .m3u, .pls, xpsf.

''Mentor: Baptiste Coudurier''

=== NEW Seeking API ===
*primary goal: implement a new seeking API in libavformat
** implement av_seek_file in libavformat
** implement compatible new seek_file for all AVInputFormat porting existing seek function if possible.
** implement av_build_index function which will build an AVIndex for the file
** implement av_export_index function which will save AVIndex in a file which can be loaded later.

''Mentor: Baptiste Coudurier''

=== Improve RTSP/RTP layer ===
*primary goal: improve the receiver compatibility
** Add support for more widespread formats ([list will follow check gst live555 and feng])
*** X-Qt/quicktime depayloader (see [http://www.gnome.org/~rbultje/ffmpeg-patchset/ X-QT patch])
*** vorbis and theora depayloader (see [[Small_FFmpeg_Tasks#Implement_the_RTP.2FVorbis_payload]])
*** h263 and h263+ (see [http://roundup.ffmpeg.org/roundup/ffmpeg/issue678 Issue 678])
*** ...more...
** support Quicktime http tunnel mode
*secondary goal: provide an API to expose the rtcp layer (and the equivalent in RDT dialect)
*secondary goal: try to support subtitle streams (either as rtcp-xr or application/text stream)
*secondary goal: make VideoLanClient, MPlayer and Xine use ffmpeg rtsp

''Mentor: Luca Barbato, Ronald S. Bultje''

== 2nd Tier Project Proposals ==
All that separates these proposals from their 1st tier brethren is a mentor.

=== [[Libavfilter]] video work ===
Libavfilter is the FFmpeg filtering library that started as a 2007 SoC [[FFmpeg Summer Of Code#Video Filter API (AKA libavfilter)|project]]. It should replace the now removed vhook subsystem. Most of it is already part of the FFmpeg main source tree, but there a few bits remaining. This project would consist in the following tasks

* Get the remaining bits of the SoC tree committed, including the ffmpeg.c and ffplay.c patch
* Get libavfilter enabled in the main SVN tree
* Write a watermark filter (this is one of the most commonly requested FFmpeg feature)
* Write a expand/pad filter (see [http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/85015] and [http://thread.gmane.org/gmane.comp.video.ffmpeg.soc/2779/]
* Port all MPlayer filters at libmbcodec/vf_* (do not forget asking the authors if it is ok to release them under the LGPL)

see also this ffmpeg-devel message: [http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-March/064817.html]

=== [[Libavfilter]] audio work ===
At the moment, FFmpeg filtering library has no support at all for handling audio. This task would consist of

* Expanding the libavfilter framework to work with audio
* Writing a resampling filter (starting with just a wrapping code at libavcodec/audioconvert.c)
* Implement negotiation of sample format and number of channels analogously to the libavfilter colorspace negotiation
* Make the resampling filter works for several combinations of sample format and channels
* Write a visualization filter as proof-of-concept of a filter that works with both video and audio

=== Implement a better regressions test system ===
* Split up the current regtests
* Add tests for all the missing formats and codecs to FATE

=== libvo ===
* Port MPlayer's libvo to ffplay
* Note that this does not just mean to produce a working hack so that ffplay can use xv, but a clean and acceptable wrapper for (most of) libvo.

=== GStreamer input ===
* Like we have vfw input we could have a gstreamer input format also. This would enable support of wmapro and wmalossless until these formats are RE'd.
''Mentor: Christian Schaller''

=== AMR-WB Decoder ===
* Specification: http://www.3gpp.org/ftp/Specs/html-info/26-series.htm
* Samples: http://samples.mplayerhq.hu/A-codecs/amr/
Also see [[AMR]].

=== GSM Decoder ===
* Specification + sample implementation: http://kbs.cs.tu-berlin.de/~jutta/toast.html
* Samples: http://samples.mplayerhq.hu/A-codecs/GSM/
Also see [[GSM]].

=== Sipr Decoder ===
* Specification: will be provided
* Samples: http://samples.mplayerhq.hu/real/AC-sipr/
Also see [[RealAudio sipr]] and [[Interesting_Patches#RealAudio_SIPR_.4016k_decoder_and_demuxer_by_Vladimir_Voroshilov|this patch]].

=== Speex Decoder ===
* Specification: http://speex.org/docs/
* Samples: http://samples.mplayerhq.hu/A-codecs/speex/
Also see [[Speex]].

=== AMR-NB Encoder ===
* Specification: http://www.3gpp.org/ftp/Specs/html-info/26-series.htm
* Samples: http://samples.mplayerhq.hu/A-codecs/amr/
Also see [[AMR]].

=== VP6 Encoder ===
* Specification: [[On2 VP6]]
* Samples: http://samples.mplayerhq.hu/V-codecs/VP6/

=== WMV3 Encoder ===
* Clearly defined task
* Primary goal: Encode video sequences such that they can be decoded by a Windows Media player.

This could either be done by improving [[Interesting Patches#WMV3 encoder by Denis Fortin|this patch]] or by writing the encoder from scratch.

=== Improve subtitle support ===

* Add text-to-bitmap conversion functions
* One with hard-coded bitmaps for characters
* One that utilizes freetype
* Function used will be chosen upon compilation

Adjust existing subtitle support to new ABI
* ABI change: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-January/058521.html

=== AACS implementation ===
* Add the ability to encode and decode using Advanced Access Content System to FFmpeg.
* Specifications: http://www.aacsla.com/specifications/
* Most parts (BD-J, MKB, title key generation) probably do not belong into FFmpeg, this should be discussed with us before submitting an application

''Mentor: Reimar Döffinger''

=== VC-1 Interlaced Support ===
* Add support for interlaced streams as used in Bluray recordings to the VC-1 decoder.
* This includes fixing some reference streams

=== Improve Ratecontrol ===
*Primary goal 1: Fast heuristic VBV compliant per macroblock ratecontrol which has a better PSNR/bitrate and better subjective quality/bitrate than the current code.
*Primary goal 2: VBV compliant, rate distortion optimal per macroblock ratecontrol using the viterbi algorithm.
*Secondary goal 1: Fast heuristic scene change detection which detects scene changes more accurately, has better PSNR/bitrate and subjective quality/bitrate than the current heuristic.
*Secondary goal 2: Rate distortion optimal (for the current picture) scene change detection.
*Secondary goal 3: B frames decision which is faster and or has a higher PSNR/bitrate and subjective quality/bitrate than the current code.

=== WMA lossless ===
* Implement a decoder for WMA lossless (0x0163)
* Reuse as much libavcodec code as possible
* Samples: http://samples.mplayerhq.hu/A-codecs/WMA9/wma_0x163.wma http://samples.mplayerhq.hu/A-codecs/lossless/luckynight.wma

=== WTV (de)muxer ===
* Implement a demuxer (and possibly a muxer) for the [[WTV]] file format.

Talk:FFmpeg Summer Of Code 2009

2009-03-19T18:26:48Z

Reimar: comments on libvo/AACS tasks

=== S/PDIF muxer ===

Is there any specific qualification task you would like done for this? -- Jai

:Working Jpeg2000 decoder ;), cleaning up this http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2005-June/001673.html would be welcome. It's a rpza encoder. --[[User:Merbanan|Merbanan]] 06:22, 31 December 2008 (EST)

=== speex + gsm ===

Aren't libgsm and libspeex distributed under a permissive license?
If yes, these tasks do not have very high priority, imo.
[[User:Ce|Ce]] 14:56, 11 January 2009 (EST)

: That's not important. FFmpeg aims to support all multimedia formats. We have never let this argument stop us from implementing decoders. --[[User:DonDiego|DonDiego]] 08:22, 14 March 2009 (EDT)

=== DTS-HD Master Audio decoder? ===
Would [http://en.wikipedia.org/wiki/DTS-HD_Master_Audio DTS-HD Master Audio] decoder make good project suggestion? [[User:Gamester17|Gamester17]] 02:51, 16 January 2009 (EST)

http://en.wikipedia.org/wiki/DTS-HD_Master_Audio
:""''DTS-HD Master Audio is a lossless audio codec created by Digital Theater System. It was previously known as DTS++ and DTS-HD. It is an extension of DTS which, when played back on devices which do not support the Master Audio extension, degrades to a 1.5 Mbit/s "core" track which is lossy. DTS-HD Master Audio is an optional audio format for both Blu-ray Disc and HD DVD''"

Specs, please. From what I know the projects without spec take a looong time to complete. --[[User:Kostya|Kostya]] 03:32, 16 January 2009 (EST)

:AFAIK, there is even no software implementation, so it would be even more difficult;-( [[User:Ce|Ce]] 20:14, 16 January 2009 (EST)

::How about then qualification task for at least distinguishing between normal DTS and DTS-HD Master Audio, to let the users know that the audio stream DTS-HD Master Audio but they are only getting normal DTS output out of it from FFmpeg? As today FFmpeg reports all as just "''dca''", (I understand that [http://mediainfo.sourceforge.net MediaInfo] is an open source C++ project that is capable of distinguishing between normal DTS and DTS-HD Master Audio. [[User:Gamester17|Gamester17]] 12:20, 21 January 2009 (EST)

=== WTV (Microsoft Windows Media Center Recording Format) demuxer? ===
Would a [[WTV|WTV (Microsoft Windows Media Center Recording Format)]] demuxer make good project suggestion? [[User:Gamester17|Gamester17]] 13:14, 16 January 2009 (EST)

[[WTV|http://wiki.multimedia.cx/index.php?title=WTV]]
:"''WTV is the new container format used to record television shows in Microsoft Windows Vista Media Center starting with Windows Media Center TV Pack 2008.''", "''WTV is the successor of DVR-MS which is is being replaced with WTV''", "''WRT is also the default recording format for Windows 7 Media Center''"

::This is tricky. It doesn't strike me as being involved enough to qualify as one of our usual SoC projects. OTOH, it seems a little too involved to be a qualification task. --[[User:Multimedia Mike|Multimedia Mike]] 14:24, 16 January 2009 (EST)

::: That sounds great actually. Maybe this could become one of the few SoC projects that are actually finished in time...--[[User:DonDiego|DonDiego]] 07:18, 18 January 2009 (EST)

:::: +1! [[User:Ce|Ce]] 11:54, 18 January 2009 (EST)
:Added to article --[[User:Ce|Ce]] 12:56, 19 March 2009 (EDT)

=== libavui (a common skins library)? ===
Would a common skins library make good project suggestion?
*MPlayer skin
*VLC skin
*Xine skin
*XMMS skin
*WINAMP skin
*Windows Media Player skin
*Rockbox skin
*foobar2000 skin
*Songbird feathers (skin)
-[[User:Nazo|Nazo]] 21:29, 16 January 2009 (EST)
: Personally, I would advocate a project to stamp out skinnable UIs across the computing landscape. But that's outside of the scope of an SoC project. I hate UI skins. --[[User:Multimedia Mike|Multimedia Mike]] 14:03, 17 January 2009 (EST)
:: I second that. But I don't see how GUI stuff like promoting or discouraging skins relates to libav* in the first place. [[User:Koorogi|Koorogi]] 16:26, 17 January 2009 (EST)
: No, skins are outside the scope of FFmpeg.--[[User:DonDiego|DonDiego]] 07:18, 18 January 2009 (EST)

==Refactor VDPAU patch for video editing ==
This might be a good project: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-January/059032.html but I don't know for sure, so that's why I am including it on this page. [[User:Dashcloud|Dashcloud]] 16:23, 20 January 2009 (EST)

==More pixel format support?==
Would more pixel format support make good project suggestion? Here is crazy missing pixel format list (from HDPhoto):
*1/2/4bpp palette - 8bpp is already supported
*1/2/4/32bpp gray - 8bpp and 16bpp are already supported
*16bpp gray fixedpoint
*32bpp gray float
*48/96bpp RGB - 24bpp is already supported
*48/64bpp RGB half
*48/64/96/128bpp RGB fixedpoint
*32bpp RGB101010
*96/128bpp RGB float
*64/128bpp RGBA - 32bpp is already supported
*64/128bpp RGBA fixedpoint
*64bpp RGBA half
*128bpp RGBA float
*32bpp BGR - 24bpp is already supported
*32bpp PBGRA
*64bpp PRGBA
*128bpp PRGBA float
*32bpp RGBE
*32/64bpp CMYK
*40/80bpp CMYKAlpha
*12bpp YUV420
*16bpp YUV422
*24bpp YUV444
*24bpp 3Channels
*32bpp 4Channels
*40bpp 5Channels
*48bpp 6Channels
*56bpp 7Channels
*64bpp 8Channels
*48bpp 3Channels
*64bpp 4Channels
*80bpp 5Channels
*96bpp 6Channels
*112bpp 7Channels
*128bpp 8Channels
*32bpp 3ChannelsAlpha
*40bpp 4ChannelsAlpha
*48bpp 5ChannelsAlpha
*56bpp 6ChannelsAlpha
*64bpp 7ChannelsAlpha
*72bpp 8ChannelsAlpha
*64bpp 3ChannelsAlpha
*80bpp 4ChannelsAlpha
*96bpp 5ChannelsAlpha
*112bpp 6ChannelsAlpha
*128bpp 7ChannelsAlpha
*144bpp 8ChannelsAlpha
--[[User:Nazo|Nazo]] 07:24, 21 January 2009 (EST)

: See [[Small FFmpeg Tasks#Generic Colorspace system]]. [[User:Vitor|Vitor]] 14:11, 22 January 2009 (EST)
:: need samples for each one at least. --[[User:Compn|Compn]] 07:33, 23 January 2009 (EST)
::: and HDPhoto decoder --[[User:Kostya|Kostya]] 08:06, 23 January 2009 (EST)
:::: HDPhoto format is similar to tiff. HDPhoto support uncompressed image, I remembered. but I checked, it was for future use:-( --[[User:Nazo|Nazo]] 09:46, 23 January 2009 (EST)

==VC-1 Interlaced Support==
Blu-Ray media contain interlaced VC-1 which is currently not supported by lavc decoder. Since a lot of people showed the lack of interest to implement it, maybe some student will take this task. --[[User:Kostya|Kostya]] 09:23, 26 January 2009 (EST)
:Added to article --[[User:Ce|Ce]] 09:53, 4 March 2009 (CST)

==Interactive command ui support==
Interactive command ui might be good start point to implement missing features for GUI encoders.
* more presets
* containable codec in each format
* mapping between encode options and proper UI types
* playable codecs and encode options on each player (WMP, RealPlayer, Flash Player, Silverlight, PS3, mobile phones, etc...)
* list of usable metadata tags in each format
* etc...
--[[User:Nazo|Nazo]] 05:22, 28 February 2009 (CST)

==Maybe remove/downshift some tasks==
Some tasks would be hard to complete:
* Flash Screen video 2 codec - without Mike providing documentation it's a bit harder. I remember one student who was promised RV40 specs and got almost nothing; decoder was more or less completed successfully but it took slightly more time.
* GStreamer input - rather weak excuse. Better make it the task of REing WMA lossless (WMApro is almost there).
* i263 decoder - FFmpeg supports i263 to some extent. Missing bits are probably make only a small FFmpeg task.
--[[User:Kostya|Kostya]] 10:39, 4 March 2009 (CST)

I also have my doubts about libvo (typical cleanup task, that has not worked out well in the past) and the AACS task.
AACS could work out, but IMO only the "last" part, decoding when the title key has already be found fits in FFmpeg, and then I think it should at least be AACS + CSS (I hope they would be able to share some code, at least they use the same bits for flagging the encryption).

[[User:Reimar|Reimar]] 14:26, 19 March 2009 (EDT)

== ffmpeg.c factorization ==

Every once in a while, someone shows up at ffmpeg-devel with a patch for turning ffmpeg.c into a library (for example [http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-January/059828.html]). While this is a bad idea for several reasons, having a lot of people asking for it shows that it is very hard to use libav* in a client application while been as flexible as the command-line tool.

So I suggest a SoC project of factorizing ffmpeg.c into several public, clean, well-documented functions, to simplify the use libav* and turning ffmpeg.c into a good API example.

* '''Note''': The tricky part of this project would be to factorize ffmpeg.c into functions that actually should be in a lib, not just mechanically moving code to libav*/something.c

-[[User:Vitor|Vitor]] 13:45, 4 March 2009 (CST)

== Rate control project ==
I copied this one from the 08 SoC page because I think it would be a great project- but if people think it doesn't need to be copied from last year's page, then I'll delete the entry. [[User:Dashcloud|Dashcloud]] 22:08, 4 March 2009 (CST)

What does viterbi have to do with macroblock-level ratecontrol, with or without VBV constraints? I can vaguely see viterbi on frame-level VBV compliance, using "remaining VBV space" as the state to be trellised over, if you're willing to quantize possible frame sizes enough to bring the number of states down to something sane. But that isn't relevant to macroblocks: VBV doesn't impose any constraints smaller than a frame, so macroblock-level is plain old independent RDO. [[User:Pengvado|Pengvado]] 08:33, 13 March 2009 (EDT)

== WMA lossless ==

I'm not sure this would qualify as a project unless someone (generally other than the student, unless he is really gifted) takes the responsibility of reverse engineering it

-[[User:Vitor|Vitor]] 13:50, 6 March 2009 (CST)

The situation is the same as with WMA3 lossy and we have working decoder for it.

--[[User:Kostya|Kostya]] 23:38, 6 March 2009 (CST)

== Shoutcast support ==

Is adding support for shoutcast (streaming & receiving) a viable task? -- Jai
:preliminary patch here: https://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-January/060556.html --[[User:Compn|Compn]] 11:42, 14 March 2009 (EDT)

== Playlist/Concatenation Support ==

I suspect that interface can be easily extended by two options only: -playlist X and -concat, the latter tells FFmpeg to concatenate all input files instead of processing them in parallel. The problem will be mostly in formats negotiations during concatenation (i.e. frame dimensions and rate mismatch for different files). --[[User:Kostya|Kostya]] 07:52, 19 March 2009 (EDT)

Small FFmpeg Tasks

2009-03-17T13:19:34Z

Reimar: A demuxer for the Metal Gear Solid is a suitable small task as well

This page contains ideas for small, relatively simple tasks for the [[FFmpeg]] project. People who might be interested in trying one of these tasks:
* Someone who wants to contribute to FFmpeg and needs to find a well-defined task to start with
* Someone who wishes to qualify for one of FFmpeg's coveted [[FFmpeg Summer Of Code|Summer of Code]] project slots
* An existing FFmpeg developer who has been away from the project for a while and needs a smaller task as motivation for re-learning the codebase

For other tasks of varying difficulty, see the [[Interesting Patches]] page.

'''If you would like to work on one of these tasks''', please take these steps:
* Subscribe to the [https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel FFmpeg development mailing list] and indicate your interest
* Ask [[User:Multimedia Mike|Multimedia Mike]] for a Wiki account so you can claim your task on this Wiki

'''If you would like to add to this list''', please be prepared to explain some useful details about the task. Excessively vague tasks with no supporting details will be ruthlessly deleted.

=== Finish up a previous incomplete SoC project ===

Several SoC projects from previous years have not yet made it into FFmpeg. Taking any of them and finishing them up to the point that they can be included should make for a good qualification task. Check out the [[FFmpeg Summer Of Code]] overview page and look for the unfinished projects, like AMR-NB, Dirac, TS muxer, JPEG 2000.

=== Generic Colorspace system ===
This task involves adding support more than 8 bits per component (Y on 10 bits, U on 10 bits, V on 10 bits for example)
and generic simple conversion to other colorspaces.

''Does this have to do with revising FFmpeg's infrastructure? If so, then it doesn't feel like a qualification task. If it's something simpler, then the vague description does not convey that simplicity. Please expound.'' --[[User:Multimedia Mike|Multimedia Mike]] 12:56, 25 February 2008 (EST)

''I don't think so, extending PixFmt to extended structure with finegrained description like depth, range values, colorspace, sample period, and write generic simple conversion from all formats to all others, like suggested by Michael on the mailing list. Conversion routine can be a good qualification task for video encoders/decoders. What do you think ?
--[[User:Bcoudurier|Baptiste Coudurier]] 00:30, 29 February 2008 (EST)

''* Adding the [[YCoCg]] colorspace (with different sized planes) for RGB sourced pictures would be nice too. [[User:Elte|Elte]] 07:15, 16 March 2009 (EDT)

=== Make the SoC dts encoder multichannel capable ===
Here is a skeleton for a dts encoder http://svn.mplayerhq.hu/soc/dcaenc/, currently it can only encode stereo streams.
The task is to extend it to support 5.1 channels also.

Specs and info can be found here:
http://wiki.multimedia.cx/index.php?title=DTS

=== GIF LZW Encoder and extend Encoder and Decoder to support Animated GIFs ===

Lzw encoder is already used for TIFF, it must be extended to support GIF flavor.

=== Patch cleanup for MPEG 1 & 2 optimizations ===
Details are in the issue tracker: http://roundup.ffmpeg.org/roundup/ffmpeg/issue100

=== Implement a Vivo demuxer for FFmpeg ===
Implement an FFmpeg demuxer for the [[Vivo]] file format. The best reference for understanding the format would be MPlayer's [http://svn.mplayerhq.hu/mplayer/trunk/libmpdemux/demux_viv.c?view=markup existing .viv demuxer].

This task corresponds to issue 99: http://roundup.ffmpeg.org/roundup/ffmpeg/issue99

''I am ready to help out with understanding MPlayer's demuxer, esp. MPlayer API stuff if necessary.
--[[User:Reimar|Reimar]] 15:46, 1 March 2008 (EST)

=== Port missing demuxers from MPlayer to FFmpeg ===
MPlayer supports a few container formats in libmpdemux that are not yet present in libavformat. Porting them over and gettting them relicensed as LGPL or reimplementing them from scratch should make reasonable small tasks.

''Jai Menon is working on porting the tivo demuxer''

=== Optimal Huffman tables for (M)JPEG ===
This task is outlined at http://guru.multimedia.cx/small-tasks-for-ffmpeg/ and is tracked in the issue tracker: http://roundup.ffmpeg.org/roundup/ffmpeg/issue267
:Indrani Kundu Saha is currently working on this task as a qualification for Google SoC 2009 --[[User:Ce|Ce]] 19:41, 13 March 2009 (EDT)

=== YOP Playback System ===
This task is to implement an FFmpeg playback subsystem for [[Psygnosis YOP]] files. This will entail writing a new file demuxer and video decoder, both of which are trivial by FFmpeg standards. [[Psygnosis YOP|The Psygnosis YOP page]] contains the specs necessary to complete this task and points to downloadable samples.

=== M95 Playback System ===
This task is to implement an FFmpeg playback subsystem for [[M95]] files. This will entail writing a new file demuxer and video decoder (the audio is already uncompressed), both of which are trivial by FFmpeg standards. [[M95|The M95 page]] contains the specs necessary to complete this task and points to downloadable samples.

=== BRP Playback System ===
This task is to implement an FFmpeg playback subsystem for [[BRP]] files. This will entail writing a new file demuxer as well as a video decoder that can handle at least 2 variations of format data. Further, write an audio decoder for the custom DPCM format in the file. All of these tasks are considered trivial by FFmpeg standards. [[BRP|The BRP page]] contains the specs necessary to complete this task and points to downloadable samples for both known variations.

=== 16-bit Interplay Video Decoder ===
FFmpeg already supports [[Interplay MVE]] files with [[Interplay Video|8-bit video data]] inside. This task involves supporting 16-bit video data. The video encoding format is mostly the same but the pixel size is twice as large. Engage the ffmpeg-devel list to discuss how best to approach this task.

=== 16-bit VQA Video Decoder ===
FFmpeg already supports Westwood [[VQA]] files. However, there are 3 variations of its custom video codec. The first 2 are supported in FFmpeg. This task involves implementing support for the 3rd variation. Visit the VQA samples repository: http://samples.mplayerhq.hu/game-formats/vqa/ -- The files in the directories Tiberian Sun VQAs/, bladerunner/, and dune2000/ use the 3rd variation of this codec. The [[VQA|VQA page]] should link to all the details you need to support this format.

=== HNM4 Playback System ===
This task is to implement an FFmpeg playback subsystem for [[HNM4]] variant of the [[HNM]] format. This will entail writing a new file demuxer and video decoder, both of which are trivial by FFmpeg standards. [[HNM4|The HNM4 page]] contains the specs necessary to complete this task and links to downloadable samples.

=== Apple RPZA encoder ===
A patch was once sent to the ffmpeg-devel mailing list to include an encoder for the [[Apple RPZA]] video codec. That code can be found on the "[[Interesting Patches]]" page. This qualification task involves applying that patch so that it can compile with current FFmpeg SVN code and then cleaning it up per the standards of the project. Engage the mailing list to learn more about what to do.

=== QuickTime Edit List Support ===
Implement edit list support in FFmpeg's QuickTime demuxer (libavformat/mov.c). This involves parsing the 'elst' atom in a QuickTime file. For a demonstration of how this is a problem, download the file menace00.mov from http://samples.mplayerhq.hu/mov/editlist/ and play it with ffplay or transcode it with ffmpeg. Notice that the audio and video are ever so slightly out of sync. Proper edit list support will solve that. Other samples in that directory also presumably exhibit edit list-related bugs. The [http://xine.cvs.sourceforge.net/xine/xine-lib/src/demuxers/demux_qt.c?view=markup Xine demuxer] has support for this, it might be useful for hints.

:Krishna Gadepalli is working on this (patch submitted to ffmpeg-devel , currently in review) --[[User:Compn|Compn]] 10:35, 14 March 2009 (EDT)

=== Reimplement libavcodec/fdctref.c ===
The forward double precision DCT in this file has a non-free license. We need an LGPL replacement of this file.

=== Implement the Flash Screen Video codec version 2 ===
FFmpeg is missing both a decoder and an encoder. Would be nice to have that.

=== Add wma fixed point decoder back into libavcodec ===
http://svn.rockbox.org/viewvc.cgi/trunk/apps/codecs/libwma/
Rockbox's fixed-point WMA decoder was adapted from the decoder in libavcodec.

=== RealAudio 14.4 encoder ===
FFmpeg contains a decoder for [[RealAudio 14.4]], a farily simple integer CELP codec. Write an encoder. This would be a good qualification task for anyone interested in working on AMR, Speex, or sipr.

=== VC1 timestamps in m2ts ===

Codec copy of VC1 from m2ts currently doesn't work. Either extend the VC1 parser to output/fix timestamps, or fix the timestamps from m2ts demuxing.

=== FLIC work ===

Revise the [[Flic Video]] decoder at libavcodec/flicvideo.c to support video transported in AVI or MOV files while making sure that data coming from the usual FLI files still works. 'AFLC' and 'flic' FourCC samples are linked from the [[Flic Video]] page.

=== Hook up QT YUV2 FourCC ===

Wire up the YUV2 FourCC that can occur in [[MOV]] to the [[YUV 4:2:2]] colorspace. Samples are linked from the [[YUV 4:2:2]] wiki page.
[[Category:FFmpeg]]

=== CorePNG Decoder ===

Extend FFmpeg's PNG decoder to handle the difference frames and [[YUV]] colorspace added in [[CorePNG]]. Sample at [http://samples.mplayerhq.hu/V-codecs/PNG1/ http://samples.mplayerhq.hu/V-codecs/PNG1/]

=== Extend PNG Decoder ===

get this png working in ffpng: http://roundup.ffmpeg.org/roundup/ffmpeg/issue813 .

=== CJPG format ===

Extend FFmpeg's MJPEG decoder to handle the different frames/packing of CJPG. Samples at: http://roundup.ffmpeg.org/roundup/ffmpeg/issue777

=== Optimize Theora Decoder ===

speed up the Theora decoder. [http://www.archive.org/download/AlternativeFreedom/alternative_freedom.ogg 720:480 sample] hits 100% cpu on a p4 1.5ghz.
:''Do you have any specific optimizations tips? I like these small tasks to present a clearer jumping-off point. --[[User:Multimedia Mike|Multimedia Mike]] 18:57, 22 December 2008 (EST)''
::''did theora make use of the mmx/sse functions of ffvp3? i was looking at the xiph GSOC page which mentioned a similar task. --[[User:Compn|Compn]] 21:17, 22 December 2008 (EST)''
::''The major optimization I can think of is reworking coefficient decoding to avoid the continue in unpack_vlcs() (basically by having a list of coefficient VLCs for each position rather than for each block, then decoding them when actually rendering the block.) Unfortunately this also requires reworking render_slice() and reverse_dc_prediction() quite significantly which is why I haven't done it yet. [[User:Yuvi|Yuvi]] 18:25, 23 December 2008 (EST)''

=== flip flag for upside-down codecs ===

<pre>about the flip, a patch that decodes images fliped when
codec_tag == ff_get_fourcc("GEOX") is welcome.
its a metter of 2lines manipulating data/linesize of imgages after
get_buffer() or something similar
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
</pre>
more info:
http://roundup.ffmpeg.org/roundup/ffmpeg/issue741

=== lavf-based concatenation tool ===

Unless we have multiple files input in FFmpeg, it would be nice to have some libavformat-based tool that would extract frames from multiple files (possible different containers as well) and put them into single one.

=== cljr and vcr1 encoders ===
According to this: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-February/063647.html both of the encoders are disabled, and won't compile if enabled. Michael would prefer to keep them around, and have someone grow them into full encoders.

=== Add waveformat extensible support in wav muxer ===
http://article.gmane.org/gmane.comp.video.ffmpeg.devel/79503
Clean up that patch.

=== implement some colorspace fourcc/codecs ===
some colorspace formats were uploaded to http://samples.mplayerhq.hu/V-codecs/
including:
2vuy.avi
CYUV.AVI
P422.AVI
UYNV.AVI
UYNY.avi
V422.AVI
YUNV.AVI
a12v.avi
auv2.avi
and V-codecs/yuv8/MAILTEST.AVI .

step by step tutorial for adding new input formats to swscale:
cd mplayer/libswscale/
svn di -r20426:20427
the hunks 3 and 5 you dont need, they are optional special converters
also the change to isSupportedOut() you dont need
above will add a new input format

another example for adding an input format
cd mplayer/libswscale/
svn di -r20604:20605

=== Create a libamr compatible library of the Android amr codec ===
http://android.git.kernel.org/?p=platform/external/opencore.git;a=tree;f=codecs_v2/audio/gsm_amr/amr_nb;h=4bac3ee5bd1ae8b6955f2d0bdac7de43c0d985c1;hb=HEAD

=== Make the rtp demuxer support rtcp BYE packets ===
rtcp BYE (203) packets are sent from the sender to the receiver to notify that a stream has ended.
FFmpeg currently ignores them.

Sample url rtsp://media.lscube.org/tests/tc.mov

=== Implement the RTP/Vorbis payload ===
This is supported by the [http://www.lscube.org/projects/feng feng RTSP server], and is described in [http://tools.ietf.org/html/rfc5215 RFC 5215]. For testing, you can set up a local feng RTSP server to stream some local Vorbis file, or you can use the online feng test-server (rtsp://media.lscube.org:554/tests/rms_profumo_1.ogv).

Most likely, your implementation will consist of a file called rtp_vorbis.c in libavformat/, which will read the header packets available in the SDP (the "configuration" piece in the fmtp: line) and which parses individual incoming RTP packets from the RTSP demuxer (minus the generic RTP header bits). It should output Vorbis-encoded frames which can subsequently be decoded by the Vorbis decoder in libavcodec/.

=== Implement the RTP/Theora payload ===
The Theora payload is currently still a [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt draft]. Yet, it would be nice to support this payload. As per above, the [http://www.lscube.org/projects/feng feng RTSP server] supports the Theora RTP payload draft and can be used for testing your implementation of the draft, or you can use the online feng test-server (rtsp://media.lscube.org:554/tests/rms_profumo_1.ogv).

Most likely, your implementation will consist of a file called rtp_theora.c in libavformat/, which will read the header packets available in the SDP (the "configuration" piece in the fmtp: line) and which parses individual incoming RTP packets from the RTSP demuxer (minus the generic RTP header bits). It should output Theora-encoded frames which can subsequently be decoder by the Theora decoder in libavcodec/.

=== cdg decoder + demuxer ===
create a [[CD Graphics]] decoder/demuxer. implementations: http://www.kibosh.org/pykaraoke/ or http://users.fbihome.de/~glogow/ or http://miageprojet.unice.fr/twiki/bin/view/Fun/JavaKarPlayer or http://www.kibosh.org/cdgtools/ or this http://bat-kolio.net/cdg2video/ (which uses ffmpeg).

=== support for YCoCg/RGB colospace in FFV1 ===
add support for [[YCoCg]] and [[RGB]] encoded sources

This would add a free lossless intra-frame RGB codec for all by FFmpeg supported platforms (most important MacOS + Windows) which is often asked for video editing in video forums (e.g. slashcam.de)

=== Metal Gear Solid Video format demuxer ===
http://multimedia.cx/eggs/metal-gear-vp3/

V210

2009-02-17T19:33:45Z

Reimar:

[http://developer.apple.com/quicktime/icefloe/dispatch019.html#v210 v210 QuickTime Description]

The v210 format is a packed YUV 4:2:2 (UYVY) format with 10 bits per component.

Data is stored in blocks of 32 bit values in little-endian.

Each such block contains 3 components, one each in bits 0 - 9, 10 - 20 and 20 - 30, the remaining two bits are unused.

Since UYVY consists of 4 components that represent two pixels, which of those bits correspond to which component makes a pattern that repeats every 4 32-bit blocks:

block 1, bits 0 - 9: U0+1
block 1, bits 10 - 20: Y0
block 1, bits 20 - 30: V0+1
block 2, bits 0 - 9: Y1
block 2, bits 10 - 20: U2+3
block 2, bits 20 - 30: Y2
block 3, bits 0 - 9: V2+3
block 3, bits 10 - 20: Y3
block 3, bits 20 - 30: U4+5
block 4, bits 0 - 9: Y4
block 4, bits 10 - 20: V4+5
block 4, bits 20 - 30: Y5

In addition the start of each line is aligned to a multiple of 128 bytes, where unused blocks are padded with 0.

Unused parts of a partial (i.e. last) block do not seem to contain any specific value.

FFmpeg does not support this format since there is no support for any 10-bit per component pixel formats.
Patches that simply discard the two least significant bits are referenced [http://wiki.multimedia.cx/index.php?title=Interesting_Patches#v210_decoder_patches here]

V210

2009-02-17T18:34:41Z

Reimar:

The v210 format is a packed YUV 4:2:2 (UYVY) format with 10 bits per component.

Data is stored in blocks of 32 bit values in little-endian.

Each such block contains 3 components, one each in bits 0 - 9, 10 - 20 and 20 - 30, the remaining two bits are unused.

Since UYVY consists of 4 components that represent two pixels, which of those bits correspond to which component makes a pattern that repeats every 4 32-bit blocks:

block 1, bits 0 - 9: U
block 1, bits 10 - 20: Y
block 1, bits 20 - 30: V
block 2, bits 0 - 9: Y
block 2, bits 10 - 20: U
block 2, bits 20 - 30: Y
block 3, bits 0 - 9: V
block 3, bits 10 - 20: Y
block 3, bits 20 - 30: U
block 4, bits 0 - 9: Y
block 4, bits 10 - 20: V
block 4, bits 20 - 30: Y

In addition the start of each line is aligned to a multiple of 64 (or possible 128) bytes, where unused blocks are padded with 0.

Unused parts of a partial (i.e. last) block do not seem to contain any specific value.

FFmpeg does not support this format since there is no support for any 10-bit per component pixel formats.
Patches that simply discard the two least significant bits are referenced [http://wiki.multimedia.cx/index.php?title=Interesting_Patches#v210_decoder_patches here]

Interesting Patches

2009-02-17T18:33:30Z

Reimar: /* v210 decoder patches */

This page tries to collect some useful patches for FFmpeg that didn't make into SVN for some reason or another.

== native [[Zlib]] decoder by Mans Rullgard ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-July/032820.html

In the same thread, there are patches to use the native decoder in several FFmpeg decoders.

== [[WMV3]] encoder by Denis Fortin ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-June/031699.html

== [[H.263]] rtp patch ==
http://www.voxgratia.org/bin/ffmpeg-0.4.7.patch.zip, originally at http://www.salyens.com/downloads/index.html#ffmpeg-0.4.7, now removed.

== [[Apple RPZA]] encoder by Todd Kirby ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2005-June/001673.html

== Test Pattern Generator Demuxer by Nicholas George ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-October/036838.html

== Test Pattern Generator Demuxer by [[User:Angustia|Ramiro Ribeiro Polla]] ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-April/028226.html
Or
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/49447

== PES packetizer by Xiaohui Sun ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-September/034849.html

Part of the work of [[FFmpeg Summer Of Code#TS Muxer|Summer Of Code TS Muxer]]

== Imlib2script: a scriptable vhook by [[User:Wzrlpy|Víctor Paesa]] ==
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/52341

== vf_imlib2: a libavfilter filter by [[User:Wzrlpy|Víctor Paesa]] ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-soc/2007-December/002162.html

== File concatenation by Wolfram Gloger ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-July/032131.html

== "mem" file protocol by Lagrange Multiplier ==
The "mem" protocol simply uses RAM as a source for input multimedia data, akin to how the "file" and "pipe" protocols use filesystem files and pipes as sources.

http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-May/028489.html

== Presets/profiles for usual targets by Panagiotis Issaris ==
Allow to keep in a text file groups of command options, and apply them at once by specifying the target name.

Handy for iPod, PSP, or any other picky multimedia player that otherwise requires lengthy command lines.

http://article.gmane.org/gmane.comp.video.ffmpeg.devel/37244

== PiP (Picture in Picture): a vhook filter by Mihail Stoyanov ==
http://article.gmane.org/gmane.comp.video.ffmpeg.devel/38896

== [[AMV]] encoder ==
http://code.google.com/p/amv-codec-tools/

See this post [http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-October/037356.html] to see what is missing to get it into SVN.

== [[Electronic Arts Formats]] demuxer/decoder by [[User:Suxen drol|Peter Ross]]==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-October/036938.html
The format demuxer modifications and the EA video codecs have not yet been applied to FFmpeg.

== Experimental MSVC port by Ole André Vadla Ravnås ==
Code in the [http://bazaar-vcs.org bazaar] branch at http://people.collabora.co.uk/~oleavr/OABuild/bzr/ffmpeg/

Patch at http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-March/044463.html

== H264 encoder by Jori Liesenborgs & Panagiotis Issaris ==
http://research.edm.uhasselt.be/~h264/

== DTS/AC3 in wav autodetection ==
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/49812/focus=49909
Clean up this patch and also add detection of AC3 in wav, it is similar. Samples for both can be found here: http://www.sr.se/cgi-bin/mall/artikel.asp?ProgramID=2445&Artikel=739973

== [[Bink Audio]] decoder by [[User:Suxen drol|Peter Ross]] ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-April/045346.html

Note: An updated patch is under development by [[User:DrV]] based on an updated patch by the original [[User:Suxen drol|author]].

== G722 decoder by Chas Williams ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-June/048457.html

It is basically an adaptation to FFmpeg of the [http://www.soft-switch.org/ SirenDSP] decoder.

== [[Chinese AVS]] video encoder by [[User:StefanG|Stefan Gehrer]] ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-July/033286.html

== Lossless msmpeg4v3 to mpeg4 transcoder ==
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/17074

== Fixed point cook decoder ==
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/46024
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/54008
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/54553
[[Category:FFmpeg]]

== GDI screen grabbing for Win32 ==
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/43589

There are two implementations in the thread above.

== [[RealAudio sipr|RealAudio SIPR]] @16k decoder and demuxer by [[User:Voroshil|Vladimir Voroshilov]] ==

http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-September/052961.html

Expected to work with FFmpeg r15192

== [[QCELP]] reference decoder wrapper by Moriyoshi Koizumi ==

http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2006-December/020223.html

== Proper parsing of DTS-HD MA streams ==
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-November/056526.html

== [[ACELP.net]] and G.729 decoder ==
[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-May/046518.html filters]

[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-May/046519.html pitch lag decoding]

[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-May/046520.html vectors operations]

[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-May/046521.html G.729 core]

[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-May/046522.html G.729 tables]

[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-May/046523.html G.729 postfilter]

[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-May/046524.html G.729D decoder]

== v210 decoder patches ==
[http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/54195 http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/54195]

[http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-February/062881.html http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-February/062881.html]

V210

2009-02-17T18:28:13Z

Reimar:

V210

2009-02-17T18:26:34Z

Reimar:

The v210 format is a packed YUV 4:2:2 (UYVY) format with 10 bits per component.
Data is stored in blocks of 32 bit values in little-endian.
Each such block contains 3 components, one each in bits 0 - 9, 10 - 20 and 20 - 30, the remaining two bits are unused.
Since UYVY consists of 4 components that represent two pixels, which of those bits correspond to which component makes a pattern that repeats every 4 32-bit blocks:
block 1, bits 0 - 9: U
block 1, bits 10 - 20: Y
block 1, bits 20 - 30: V
block 2, bits 0 - 9: Y
block 2, bits 10 - 20: U
block 2, bits 20 - 30: Y
block 3, bits 0 - 9: V
block 3, bits 10 - 20: Y
block 3, bits 20 - 30: U
block 4, bits 0 - 9: Y
block 4, bits 10 - 20: V
block 4, bits 20 - 30: Y

In addition the start of each line is aligned to a multiple of 64 (or possible 128) bytes, where unused blocks are padded with 0.
Unused parts of a partial (i.e. last) block do not seem to contain any specific value.