FFmpeg Summer of Code 2012
FFmpeg Summer of Code Proposals and Qualification Tasks.
March 12th-16th: Project application evaluation.
Find us on irc, server: irc.freenode.net channel: #ffmpeg-devel or contact us by subscribing to the mailing list https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel/
To be eligible for a Summer of Code project, we ask you to do a small programming task to prove you know the basics. FFmpeg is a large, complicated collection of code and it's not easy for beginners. There are some ideas for tasks on the Small FFmpeg Tasks page.
1st Tier Project Proposals
These are proposals with a mentor attached.
Baptiste has also offered to mentor.
Add OpenGL output to ffplay, this should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which you may relicense under the LGPL.
Mentor: Reimar Döffinger
Improve the audio resampling/rematrixing/converting code
- Right now, we're using libswresample to resample/rematrix audio (samplerate / channels) and to resample the audio format (int, float, 16-bit, 32-bit).
- Both interleaved and planar audio sample formats are already supported
- We need SIMD optimization of popular conversions (float-int16, int16-float), (stereo-mono-5.1) and anything else that's frequently used.
- We need support for alternate conversion functions (e.g. sample format conversion with or without dithering)
- Fix bugs in current design (none known but there sure are some)
Mentor: Michael Niedermayer
Implement a H.265 / High efficiency video coding (HEVC) decoder
- Write a basic decoder supporting I, P, and, if time permits, B slices.
- It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.
- As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.
- A draft spec is available at: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/8_San%20Jose/wg11/JCTVC-H1003-v21.zip
Mentor: Michael Niedermayer
- Add MVC support to our H.264 decoder. MVC is used in 3D Blu-Rays.
- As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.
Mentor: Michael Niedermayer
Libavfilter is the FFmpeg filtering library that started as a 2007 SoC project. It currently supports audio and video filtering and generation support.
The task would consist of writing or porting audio and video filters and eventually fix/extend libavfilter API and design.
In particular the work may focus on porting MPlayer filters which are currently integrated through the mp wrapper. For each port the student should verify that the new filter produces the same output (by comparing the output generated by -vf mp=FILTER and -vf FILTER) and checking that the new integrated filter is not slower.
Prerequisites: good C coding skills, familiarity with git/source code control systems, having some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.
Qualification task: a port or a new implementation of one or more filters.
Mentor: Stefano Sabatini - saste on IRC (possibly with a backup mentor).
Bayer colorspace formats
Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmepg's applicability to RAW still and video photography processing. Tasks:
- Implement bayer transformations in libswscale (plain C)
- Add bayer formats to the libavutil pixfmt enumeration routines
- Extend TIFF decoder to support DNG-Bayer format
- Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)
- SIMD optimizations of the libswscale transformations
- Decoders/specs may be available in the Dcraw project
Qualification task: TBD
Mentor: Peter Ross
Extend image formats support
Improve FFmpeg support for image formats, adding missing formats (e.g. XPM) and extending support for the current ones (e.g. animated GIF, GIF compression, fix PNG todos, add support to animated PNG) etc.
Qualification task: TBD (possibly finally fixing and integrating Måns' zlib decoder that has been unmerged since ages? Or just starting with some small part of the task itself, or implementing format autodetection for imagepipe demuxer)
Mentor: Reimar Döffinger
- ETSI released specifications (http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). Your job is to complete the existing decoder with the following features.
(1) Add support for mixed Core + DTS-HD stream structure (DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main and commentary tracks. (2) Add support for XXCh extension (6.1 and 7.1 channels). (3) Add support for X96 extension (96khz). (4) Add support for XLL extension (lossless). (5) Add support for a pure DTS-HD stream structure (DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks. (6) Add support for XBR extension (extra bitrate).
Mentor: Benjamin Larsson
Currently libavdevice use libavformat API. This is far from being useful for real-time video rendering. Your task is to write new API which resolves this and similar issues and port all already present input and output devices to the new API.
New API should have:
* Support for output video devices which do not provide own scaler. * Minimal latency.
Qualification task: write one new video outdev, for example any of, but not limited to: aalib, caca, x11, xv, xover, vesa or gl2 using current libavdevice API.
Mentor: Paul B Mahol
Redesigning the protocol reads
Eliminate active polling while allowing interrupts, and allowing selection on multiple handles.
Mentor: Nicolas George
2nd Tier (need mentor) Project Proposals
Some of the following proposals are also proposed by other organizations, we will try to coordinate this with them so as to avoid duplicate work. We are also happy to hear your personal project ideas ...
AAC decoder improvements
Our AAC decoder does not support low-delay. Part of this task will be to also finish last year's BSAC task. A possible qualification task is to fix a crash in the current BSAC code with one of the samples from the BSAC testing suite.
AAC encoder improvements
Our AAC encoder does not produce competitive quality per bitrate. Improve the encoder to be better than some other commonly used encoder like libfaac. This requires solid understanding of things like psychoacoustics and rate distortion. A qualification task for this could be to improve the encoder by at least 5% bitrate at the same quality measured by some objective measure.
Write a system like FATE that fuzzes and tests these fuzzed multimedia files under address sanitizer and valgrind with ffmpeg and ffplay. When a crash, or other anomaly is found, it would use git bisect to identify which exact commit introduced the bug. And either display this via some web frontend (similar to fate.ffmpeg.org) or just automatically send an email to some dedicated mailing list. The system has to be robust (there will be infinite loops, OOM conditions and randomly occurring crashes). Its also important that the system is easy to maintain and can filter out duplicates of the same issue.
FFmpeg has code for interlaced VC1, but nearly all samples still do not decode correctly. The task is to finish last year's project. You should be able to find a possible qualification task by testing interlaced samples.
MPEG-4 ALS Roundup
This task is to update and enhance the existing ALS decoder as well as integrate and enhance the rudimentary encoder found at: https://github.com/justinruggles/FFmpeg-alsenc
Possible features are:
- implement rls-lms in the decoder
- do correct channel layout/sort handling in the decoder
- update to current master
- use codec private options
- implement encode2(), setting pts and duration
- document options and examples in encoders.texi
- come up with a good set of encoding tests for FATE
- implement mcc/channel sort in the encoder
- implement rls-lms in the encoder
- implement float support
Fix and improve FFserver
FFserver has been part of FFmpeg since a long time but due to lack of a motivated maintainer its a bit buggy. For this project you would have to debug and fix many bugs. It requires good skills at reading and understanding other peoples code. As a qualification task you will have to write functioning regression tests for FFserver which implicates some bugfixing to make ffserver produce the same output on all supported platforms.
Reverse engineer TAK codec and write decoder for it
TAK format is partially already documented. You need to revisit that documentation and update and/or fix any missing/wrong information. Reverse engineer codec and write working bitexact decoder.
Support for more subtitle formats
We have libass support now, either a parser (from mplayer) to convert subs into ass or something else.
MKV ordered chapters / playlist support
Get playlist stuff into ffmpeg. Playlist is blocking a few things like quicktime edit list and .asx / .pls files.
Adobe fragmented http in/out
Adobe has a new streaming format.
libavfilter 9/10bit support
Make filters work with higher bitrate codecs/colorspaces
Fix copying video between formats
Lots of h264 streams in flv, mp4, mkv, mpegts. People wish to remux these into various formats so it works on their hardware (ps3, ipod etc).
https://ffmpeg.org/trac/ffmpeg/ticket/796 https://ffmpeg.org/trac/ffmpeg/ticket/822 https://ffmpeg.org/trac/ffmpeg/ticket/954 https://ffmpeg.org/trac/ffmpeg/ticket/976
Extend paletted format support
Cleanup framework for handling better with paletted format, write a posterize filter, add support to libswscale palette output (possibly making use of libavcodec/elbg), add support for reading and saving a palette to a file and apply them to the input video (e.g. by creating ad-hoc filters).
Port formats/colorspace support from dcraw or make dcraw wrapper
Dcraw supports many raw camera formats that ffmpeg may not. Port or make a wrapper for this project.