Libav Summer Of Code 2012
How it works
Google's Summer of Code program is simple: you (the student) work on a project, full-time, during the whole summer, and you get assistance (advice, mentoring) from a seasoned Libav developer who knows his way around the project and has considerable standing in the community. By doing so, you'll learn to operate in an opensource project, you'll get relevant coding experience, and you'll have a chance at earning money while doing fun stuff during the summer. So, you need a project, a mentor, do a qualification task (see below) so we can quickly assess how good a candidate we feel you'll be for the program, and then you can apply.
Selecting a project
Below, you'll find two lists of projects:
- Projects with a mentor
- Projects without a mentor
If you choose a project with a mentor, talk to that mentor (see below) and select a suitable qualification task. Once completed, you're eligible for participating in our Summer of Code program. If you choose a project without a mentor, your first job is to find a mentor (see below). Then, once you've found a mentor, continue as before. If you don't like any of the projects, you're free to define your own project and find a mentor as mentioned before (see below for caveats).
Once you've found a project (with or without mentor), start talking to the developers of the Libav project. We can often be found on IRC, and you can talk to us on mailinglists also. Hop on irc.freenode.net channel #libav-devel, or talk to us on firstname.lastname@example.org. Here, you'll be able to ask around for mentors for projects without a mentor if you need to. If you're trying to define your own project, explain (with reasonable amount of detail) what you intend to achieve and why you think your project should be in our Summer of Code program. Once you've found a mentor, you're good to start your qualification task.
Note that the self-selected mentor needs to have considerable standing in the community to be eligible for mentoring. Likewise, if you choose to define your own Summer of Code project, some community members of considerable standing need to vouch for your project.
Your qualification task
The goal of a qualification task is to see if the mentor and student feel that, together, they will be able to finish the project of their choice. More specifically, the mentor will want to test whether the student has the skills and work ethics to complete a large coding project in a limited amount of time. The student will want to make sure that the mentor provides (useful) assistance when necessary. Therefore, students should select a mentor and a Summer of Code project before starting their work on a qualification task. The qualification task is often (but not necessarily) related to the selected project. For example, if your project will be to reverse engineer a new video codec, the qualification task may be to write a partial bitstream parser for that codec. If your project is to write a muxer for a container format, the qualification task may be to write the code to write the stream header.
There will be a second qualification task for every student: Pick a file of moderate size and reformat it in proper K&R style. The goal of this task is twofold: First it familiarizes students with the style that they will have to write their code in, second it demonstrates that students are able to submit patches from git and go through our review process.
While you're working on your qualification task, apply at http://www.google-melange.com/. The degree (and detail) with which you've finished your qualification task will determine how likely your project is to be selected. In the past few years, students that completely finished their qualification task always got selected as Summer of Code students, but that may vary depending on the number of spots we get assigned by Google, and the number of students that apply.
1st Tier Project Proposals
1st tier project proposals are project ideas that are reasonably well defined AND have a mentor volunteered.
Assembly Testing Framework
FFmpeg/Libav use an extensive amount of assembly to optimize video and audio decoding. For scientific applications and high-assurance systems (e.g. a spacecraft), accuracy and speed are vital. Assembly is difficult to code, and lends itself to mistakes. As FFmpeg/Libav has no framework for testing assembly functions, development of assembly is often hindered by each developer having to create his or her own tests. In addition, functions are not tested for speed. While FATE covers some functions, there are many that simply aren't tested. The samples in FATE do not cover all features in every decoder that has assembly functions (e.g. several 10-bit H.264 predict functions are not used in any FATE sample). FATE also does not test functions in isolation, which impedes assembly development. In addition, FATE might not cover all assembly versions of a function if multiple different levels of optimizations exist (e.g. MMX, MMX2, SSE, SSE2, SSSE3, SSE4, AVX, etc).
- Qualification task: describe how you will implement the system (probably based on x264's checkasm). It should test accuracy, speed, and be extensible. Also, contact me via IRC.
- Actual task: implement the system. At least all H.264 assembly functions should be covered and the framework should be easily extensible.
- The framework should work across all supported architectures and operating systems.
- The framework should measure exactly how fast an individual function is (e.g. using START/STOP_TIMER).
- The framework should be able to test functions in isolation.
- x264's checkasm can be used as a reference.
Mentor: Daniel Kang (Jumpyshoes on #email@example.com; firstname.lastname@example.org -- ping me on IRC and email me).
- HEVC is the next-generation video standard from the MPEG standards committee. Your job is to write a decoder that can playback HEVC files.
- It does not need to be SIMD-optimized, but preferably has relevant functions separated in DSP contexts so they're easy to optimize later on.
- This task will depend on the current standardisation status of HEVC. Final publication is in 2013 but there may be parts which will not change much anyway.
- Draft spec: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/8_San%20Jose/wg11/JCTVC-H1003-v21.zip
- HM (draft reference decoder) git mirror: http://hevc.kw.bbc.co.uk/git/w/jctvc-hm.git
Mentor: Ronald S. Bultje
- ETSI released specifcations (http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). Your job is to complete the existing decoder with the following features.
(1) Add support for mixed Core + DTS-HD stream structure (DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main and commentary tracks. (2) Add support for XXCh extension (6.1 and 7.1 channels). (3) Add support for X96 extension (96khz). (4) Add support for XLL extension (lossless). (5) Add support for a pure DTS-HD stream structure (DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks. (6) Add support for XBR extension (extra bitrate).
Mentor: Benjamin Larsson
MPEG-4 ALS Roundup
This task is to update and enhance the existing ALS decoder as well as integrate and enhance the rudimentary encoder found at: https://github.com/justinruggles/FFmpeg-alsenc
Possible features are:
- implement rls-lms in the decoder
- do correct channel layout/sort handling in the decoder
- update to current master
- use codec private options
- implement encode2(), setting pts and duration
- document options and examples in encoders.texi
- come up with a good set of encoding tests for FATE
- implement mcc/channel sort in the encoder
- implement rls-lms in the encoder
- implement float support
Mentor: Justin Ruggles
Implement an independent Opus decoder using the publicly-available specification at: http://tools.ietf.org/html/draft-ietf-codec-opus-11
- The reference source code should only be used as a normative document reference when required (i.e. this should not just be a port of libopus)
- Fully support Ogg/Opus mapping: https://wiki.xiph.org/OggOpus
- Handle CELT, SILK, and Hybrid modes (including transitions)
- (optional) Handle more than 2 channels
Mentor: Justin Ruggles
Adobe DNG Decoder (Basic Support)
Adobe Digital Negative (DNG) is an attempt at a universal file format for raw camera images. Most camera manufacturers have their own proprietary raw image format. Adobe provides tools for converting these to DNG with minimal or no loss of information for more reliable long-term support in a format with an open specification.
The project goal would be to add features required for basic support of DNG files. Some of these include:
- test/improve TIFF and LJPEG 16-bit decoding support
- implement both variants of JPEG-in-TIFF in the TIFF decoder
- add basic handling for Bayer CFA pixel format(s), including demosaicing
- conversion from camera colorspace to RGB
- export of DNG/TIFF/Exif metadata
- Wikipedia Article
- DNG samples can be created from other raw formats using the free DNG Converter program
- A good place to find raw camera samples is http://www.imaging-resource.com
Mentor: Justin Ruggles
On2 VP7 decoder
VP7 is a DCT-based video codec. At the moment, it seems to lack a reference implementation, but we do have a spec and may take hint from libavcodec's VP8 and VP6 decoder and libvpx. MPlayer can decode VP7 by loading a binary. To begin working on the project, one has to setup a reference decoder against which to compare our output. This can be done by either writing a wrapper for the binary or (maybe) hacking libvpx itself.
You might want to discuss with us how and where to start. Drop by on IRC if you need help. It is not as difficult as it sounds.
Mentor: Mashiat Sarker Shakkhar
RTMP[E|S|T|TE] protocol implementation
Currently librtmp is required for RTMPE, RTMPS, RTMPT, RTMPTE. The goal of this project is to implement these protocol variants natively in libavformat.
spin off build system into a separate project
Our build system is neat enough to make into a more general solution to be reused by other projects.
The goal of this project is to achieve exactly that. Intermediate steps will be reading, understanding and documenting the current build system, refactoring parts that can be generalized further and finally making a prototype implementation for libpostproc.
You will require skills in POSIX shell, GNU Make and a firm command of English.
Mentor: Diego Biurrun
Hardware Acceleration API
Libav has two different and incomplete api to provide hardware acceleration wrapping, the project should provide a better abstraction and migrate the currently implemented ones and additional provide more.
- Draft the API (that will require knowledge of libavcodec)
- Port vaapi/hwaccel to the new API.
- Port vdpau to the new API
- Implement Freescale VPU support
- Implement TI dce support
Mentor: Luca Barbato
Libav needs a better system to serve streams, the current codebase had a number of design defect showing its age. The new avserver should be written from scratch, leveraging the knowledge piled up.
The implementation will be incrementally complex and possibly modular.
- Implement support to rtsp record/announce
- Expose RTMP and RTSP specific API (so the server won't have to use private calls)
- Write a simple rtsp, http, rtmp redirector (listen for publish/announce and rebroadcast the received streams)
- Add the capability to serve on-demand content reading from a single path
Ideally the first implementation can be made using a poll/event loop and then moved to use threads.
Mentor: Luca Barbato
Swscale Cleanup and Additions
Libswscale is an open source library for colourspace conversion. However, it requires significant cleanup to allow it to support modern features. Knowledge of x86 assembly is preferred, though it can be learnt as part of GSOC.
- Support for more interleaved colourspaces. This is useful because interleaved chroma provides improved cache coherency and would form the basis for future optimisations in libav's decoders.
- Support for the vast number of packed pixel formats (e.g r210, v210). Many are currently implemented as decoders but it is agreed that all uses of such formats are effectively colourspace conversions, they should be included in swscale. This task will involve collaborating with the student of "Adobe DNG Decoder" (if applicable) to rewrite the pixel format headers to support more complex pixel formats.
- Inline asm to yasm conversion
- RGB work (todo: expand this)
- Other general cleanup
Mentor: Kieran Kunhya (kierank)
2nd Tier Project Proposals
This is the DTS equivalent to E-AC3 but not technically related to DTS/DTS-HD. It is found in the following specification: http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). The task is to find a way of making the official decoder decode just the LBR stream (and not mix it into the main audio) and use that to verify decoder compliance of the decoder you wrote. The spec may be incomplete or require parts to be reverse engineered from the binary.