Libav Summer Of Code 2012: Difference between revisions

From MultimediaWiki
Jump to navigation Jump to search
m (→‎DTS-HD Master Audio decoder: This already is a 1st tier project)
m (fix category)
 
(30 intermediate revisions by 10 users not shown)
Line 1: Line 1:
== How it works ==
== How it works ==


Google's Summer of Code program is simple: you (the student) work on a project, full-time, during the whole summer, and you get assistance (advice, mentoring) from a seasoned FFmpeg/Libav developer who knows his way around the project and has considerable standing in the community. By doing so, you'll learn to operate in an opensource project, you'll get relevant coding experience, and you'll have a chance at earning money while doing fun stuff during the summer. So, you need a project, a mentor, do a qualification task (see below) so we can quickly assess how good a candidate we feel you'll be for the program, and then you can apply.
Google's Summer of Code program is simple: you (the student) work on a project, full-time, during the whole summer, and you get assistance (advice, mentoring) from a seasoned Libav developer who knows his way around the project and has considerable standing in the community. By doing so, you'll learn to operate in an opensource project, you'll get relevant coding experience, and you'll have a chance at earning money while doing fun stuff during the summer. So, you need a project, a mentor, do a qualification task (see below) so we can quickly assess how good a candidate we feel you'll be for the program, and then you can apply.
 
:Please detail the deadlines for applying and being a student/mentor.


=== Selecting a project ===
=== Selecting a project ===
Line 12: Line 10:


=== Contacting developers/mentors ===
=== Contacting developers/mentors ===
Once you've found a project (with or without mentor), start talking to the developers of the FFmpeg and Libav projects. We can often be found on IRC, and you can talk to us on mailinglists also. Hop on irc.freenode.net channels #ffmpeg-devel or #libav-devel, or talk to us on libav-devel@libav.org or ffmpeg-devel@ffmpeg.org. (At this point, you're probably wondering why there are 2 IRC channels and 2 mailinglists. Ask your mentor, it'll make for a good first conversation topic.) Here, you'll be able to ask around for mentors for projects without a mentor if you need to. If you're trying to define your own project, explain (with reasonable amount of detail) what you intend to achieve and why you think your project should be in our Summer of Code program. Once you've found a mentor, you're good to start your qualification task.
Once you've found a project (with or without mentor), start talking to the developers of the Libav project. We can often be found on IRC, and you can talk to us on mailinglists also. Hop on irc.freenode.net channel #libav-devel, or talk to us on libav-devel@libav.org. Here, you'll be able to ask around for mentors for projects without a mentor if you need to. If you're trying to define your own project, explain (with reasonable amount of detail) what you intend to achieve and why you think your project should be in our Summer of Code program. Once you've found a mentor, you're good to start your qualification task.


Note that the self-selected mentor needs to have considerable standing in the community to be eligible for mentoring. Likewise, if you choose to define your own Summer of Code project, some community members of considerable standing need to vouch for your project.
Note that the self-selected mentor needs to have considerable standing in the community to be eligible for mentoring. Likewise, if you choose to define your own Summer of Code project, some community members of considerable standing need to vouch for your project.
Line 23: Line 21:
=== Applying ===
=== Applying ===
While you're working on your qualification task, apply at http://www.google-melange.com/. The degree (and detail) with which you've finished your qualification task will determine how likely your project is to be selected. In the past few years, students that completely finished their qualification task always got selected as Summer of Code students, but that may vary depending on the number of spots we get assigned by Google, and the number of students that apply.
While you're working on your qualification task, apply at http://www.google-melange.com/. The degree (and detail) with which you've finished your qualification task will determine how likely your project is to be selected. In the past few years, students that completely finished their qualification task always got selected as Summer of Code students, but that may vary depending on the number of spots we get assigned by Google, and the number of students that apply.


== 1st Tier Project Proposals ==
== 1st Tier Project Proposals ==
1st tier project proposals are project ideas that are reasonably well defined '''AND''' have a mentor volunteered.
1st tier project proposals are project ideas that are reasonably well defined '''AND''' have a mentor volunteered.
=== Assembly Unit Testing Framework ===
* Libav has a lot of assembly and not enough tests for it. Your job is to write a unit testing framework for assembly.
* The framework should work across all supported architectures and operating systems.
* The framework should measure exactly how fast an individual function is (e.g. using START/STOP_TIMER).
* The framework should be able to test functions in isolation.
* x264's checkasm can be used as a reference.
* The qualification task will be to implement at least one unit test and have an idea of how to do the rest.
''Mentor: Daniel Kang'' (Jumpyshoes on #libav-devel@chat.freenode.net; daniel.d.kang@gmail.com -- ping me on IRC and email me).


=== HEVC/H265 decoder ===
=== HEVC/H265 decoder ===
Line 33: Line 40:
* It does not need to be SIMD-optimized, but preferably has relevant functions separated in DSP contexts so they're easy to optimize later on.
* It does not need to be SIMD-optimized, but preferably has relevant functions separated in DSP contexts so they're easy to optimize later on.
* This task will depend on the current standardisation status of HEVC. Final publication is in 2013 but there may be parts which will not change much anyway.
* This task will depend on the current standardisation status of HEVC. Final publication is in 2013 but there may be parts which will not change much anyway.
* Draft spec: http://www.itscj.ipsj.or.jp/sc29/open/29view/29n12612c.htm
* HM (draft reference decoder) git mirror: http://hevc.kw.bbc.co.uk/git/w/jctvc-hm.git
Note that this is a difficult project. Do not try to apply unless you already have experience with H.264 as well as reading and writing complex C code.


''Mentor: [[User:Ronald_S._Bultje|Ronald S. Bultje]]''
''Mentor: [[User:Ronald_S._Bultje|Ronald S. Bultje]]''


=== DTS-HD decoder ===
=== DTS-HD decoder ===
Line 53: Line 63:


''Mentor: Benjamin Larsson''
''Mentor: Benjamin Larsson''


=== MPEG-4 ALS Roundup ===
=== MPEG-4 ALS Roundup ===
Line 75: Line 83:
* implement float support
* implement float support


''Mentor: [[User:Jruggle|Justin Ruggles]]''
=== Opus Decoder ===
Implement an independent Opus decoder using the publicly-available specification at:
http://tools.ietf.org/html/draft-ietf-codec-opus-11
* The reference source code should only be used as a normative document reference when required (i.e. this should not just be a port of libopus)
* Fully support Ogg/Opus mapping: https://wiki.xiph.org/OggOpus
* Handle CELT, SILK, and Hybrid modes (including transitions)
* (optional) Handle more than 2 channels
''Mentor: [[User:Jruggle|Justin Ruggles]]''
=== Adobe DNG Decoder (Basic Support) ===
Adobe Digital Negative (DNG) is an attempt at a universal file format for raw camera images. Most camera manufacturers have their own proprietary raw image format. Adobe provides tools for converting these to DNG with minimal or no loss of information for more reliable long-term support in a format with an open specification.


''Mentor: Justin Ruggles''
The project goal would be to add features required for basic support of DNG files. Some of these include:
* test/improve TIFF and LJPEG 16-bit decoding support
* implement both variants of JPEG-in-TIFF in the TIFF decoder
* add basic handling for Bayer CFA pixel format(s), including demosaicing
* conversion from camera colorspace to RGB
* export of DNG/TIFF/Exif metadata


Resources:
* [http://en.wikipedia.org/wiki/Digital_Negative Wikipedia Article]
* [http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf Specification]
* DNG samples can be created from other raw formats using the free [http://www.adobe.com/products/photoshop/extend.displayTab2.html DNG Converter] program
* A good place to find raw camera samples is http://www.imaging-resource.com
''Mentor: [[User:Jruggle|Justin Ruggles]]''




Line 98: Line 137:




''Mentor: [[User:mstorsjo|Martin Storsjö]]''
''Mentor: [[User:mstorsjo|Martin Storsjö]], [[User:lu_zero|Luca Barbato]]''
 
 


=== spin off build system into a separate project ===
=== spin off build system into a separate project ===
Line 112: Line 149:
''Mentor: [[User:DonDiego|Diego Biurrun]]''
''Mentor: [[User:DonDiego|Diego Biurrun]]''


=== Hardware Acceleration API ===


Libav has two different and incomplete api to provide hardware acceleration wrapping, the project should provide a better abstraction and migrate the currently implemented ones and additional provide more.


=== Improve the audio resampling/rematrixing/converting code ===
* Draft the API (that will require knowledge of libavcodec)
 
* Port vaapi/hwaccel to the new API.
* right now, we're using libswresample to resample/rematrix audio (samplerate / channels) and to resample the audio format (int, float, 16-bit, 32-bit).
* Port vdpau to the new API
* both interleaved and planar audio sample formats are already supported
* Implement Freescale VPU support
* We need SIMD optimization of popular conversions (float-int16, int16-float), (stereo-mono-5.1) and anything else thats frequently used.
* Implement TI dce support
* We need support for alternate conversion functions (e.g. sample format conversion with or without dithering)
* fix bugs in current design (none known but there sure are some)
 
''Mentor: [[User:Michael|Michael Niedermayer]]''
 
== 2nd Tier (need mentor) Project Proposals ==
 
=== AACS implementation ===
* this task is to get libbluray integrated and working with as many discs as possible.
* Add the ability to encode and decode using Advanced Access Content System.
* Specifications: http://www.aacsla.com/specifications/
* existing implementation e.g. DumpHD: http://forum.doom9.org/showthread.php?t=123111
* Most parts (BD-J, MKB, title key generation) probably do not belong in libavcodec/libavformat, this should be discussed with us before submitting an application
** possible solution: only implement "lowest" level (decode given the correct title key) but implement CSS en- and decryption as secondary goal
 
== ideas from previous years ==
 
 
=== [[Advanced_Audio_Coding#BSAC_AAC|BSAC AAC]] Decoder ===
 
* Separate [[AAC]] Huffman decoding and dequantization from the rest of AAC decoder.
* Write a BSAC AAC decoder.
* Reuse as much of the existing AAC decoder and libavcodec as possible.
 
 
=== Libav Usage Example, Tutorial and test suite ===
 
* Prepare a series of example program of increasing complexity
** Simple remuxer
** Simple decoding loop
** Simple encoding loop
** Simple transcoding loop
** Simple producer + consumer scenario with the network protocols
** ...
** Optionally it would lead to prepare a leaner equivalent to avconf and avplay
 
* Examine when there are inconsistencies with the current API (e.g. unexpected timestamp mangling, packet splitting) and fix the issues.
 
* Integrate the example programs as components of FATE.
 
 
=== [[VC-1]] decoder missing features  ===
* Add support for interlaced streams as used in Bluray recordings to the VC-1 decoder.
* Add support for chroma/luma scaling, used in [https://roundup.ffmpeg.org/issue1477 RTP/VC1].
* Add support for slice decoding, used in [https://roundup.ffmpeg.org/issue1477 RTP/VC1].
* Add optimizations for x86-32/64 (or ARM), so that HD streams play back faster.
* This includes fixing some reference streams
 
 
 
 
=== DTS Encoder ===
* Specification: http://wiki.multimedia.cx/index.php?title=Mirror
* Sample files: http://samples.libav.org/A-codecs/DTS/
* Qualification: Extend the encoder in the SoC tree (port the float transform from the decoder, fix the LFE channel generation).
* Primary goal: Encoder that can produce multi sample rate, multi channel files and multi bit-rate. Wav and raw muxing support.
* Secondary goal: Techniques from specification implemented, optimal codebook usage, vector quantization, simple psycho acoustics.
* Third goal: Advanced psycho acoustics.
 
 
=== Hardware accelerated decoder for Android devices ===
Most Android phones supports decoding H264 via an OpenMAX decoder interface. This interface is exposed through the Stagefright
multimedia framework. The flashplugin uses this to be able to have hardware accelerated decode of H264 bitstreams from Android
user space. As many Android devices will not be upgraded to Android 2.3 or users don't want to root/reflash their firmware this
will enable a lot of devices to install a libavcodec based media player while still having hardware accelerated H264 decode.
 
 
=== Multithreading support ===
The goal of this task will be to improve the multithreading support. The task is relatively unspecific, but your basic goal will be to add multithreading support to a variety of decoders, by using existing infrastructure. Example goals could include:
* Fix problems with the libavcodec slice-based H.264 decoder threading (ask Jason for details here)
* Implement functions in the avconv/avplay binaries to autodetect the number of CPUs/cores and autodetect the number of threads to use (if unspecified)
* define ways to use both slice-based and frame-based multithreading at the same time if decoders (and files) support both
* Multithreading in other decoders that currently don't support it
* implement slice-based and/or frame-based threading in the libavcodec VP8 decoder
* implement frame-based threading in MJPEG decoder
* implement frame-based (or if you want to hurt yourself, try slice-based) multithreading in VC-1 decoder
 
 
=== [[Libavfilter]] video work ===
Libavfilter is the filtering library that started as a 2007 SoC [[FFmpeg / Libav Summer Of Code#Video Filter API (AKA libavfilter)|project]]. It replaced the now removed vhook subsystem. Most of it is already part of the main source tree, but there a few bits remaining. This project would consist in a combination of the following tasks:
 
* Get the remaining bits of the SoC tree committed (this includes: the rotate and fps filters)
* Port all the missing filters from MPlayer (do not forget asking the authors if it is ok to release them under the LGPL)
* Framework: implement dynamic-reconfiguration of the filterchain, for supporting dynamic size/format changes
* Port filters from other frameworks (mjpeg-tools, effectv, frei0r, virtualdub, vlc, etc...)
* Write wrappers for more image processing libraries and filtering frameworks (libgimp, libgraphicsmagic, weed), extend the libopencv wrapper for supporting more filters (this may need implemented float image support in libswscale)
* Write more filters (possibly starting from already posted filters which for a reason or another were never committed, e.g. concatenate, fish, 2xsai, select, lut, eval, posterize, elbg/posterize etc.)
* Implement a compose filter (suggestion for a better name?), for creating a mosaic of the input video streams
* Write a movie sink (e.g. it could be useful for implementing a display functionality, e.g. when the video is dumped to an output device)
* Write a frequency domain filter using the FFT implementation in libavcodec
* Write a Matlab/Octave/SAGE scripting wrapper (assuming it can be done)
* More ideas?
 
 
 
=== [[Libavfilter]] audio work ===
At the moment, libavfilter has incomplete support for handling audio. This task would consist of a combination of the following tasks:
 
* Fix the remaining outstanding issues and complete the audio framework integration (check saste's audio-filters branch on gitorious)
* Make the resampling filter works for several combinations of sample formats and channels
* Write a visualization filter as proof-of-concept of a filter that works with both video and audio
* Extend/improve libav* audio resampling/rematrixing/requantization capabilities (this may need to re-design the audio API, maybe it deserves a task on its own)
* Write/port basic audio filters (check for example MPlayer/libaf)
* Write wrapper for audio frameworks (sox, ladspa, ...)
 
 
=== Flash screen video (1)/2 encoder ===
 
* Enhance the encoder patch posted to ffmpeg-devel so it can be committed
* Write optimal 2-pass encoding support for the flash screen video 1 and possibly 2 encoder (dijkstra graph)
 
 
 
 
=== MXF support ===
 
* Support Clip Wrapping, Op1B, Avid proprietary files
* Support muxing DV, H.264
* Seeking support using index
 
 
=== Standards Compliant TS Muxer ===
 
* This is harder than it might seem at first because of the myriad of codecs and muxing rules.
* In order to complete this task you must find (or ask for) the spec, read it and understand the buffering models and what you will need to do for each codec. It's much easier to start with MPEG codecs to begin with. In most cases this will involve extracting or guessing information from the input stream.
* Sometimes spec-incompliant modes might be ok such as H.264 in TS files without HRD for iPod streaming.
* Later work can focus on remuxing
 
 
=== NEW Seeking API ===
*primary goal: implement a new seeking API in libavformat
** implement av_seek_file in libavformat
** implement compatible new seek_file for all AVInputFormat porting existing seek function if possible.
** implement av_build_index function which will build an AVIndex for the file
** implement av_export_index function which will save AVIndex in a file which can be loaded later.
 
 
=== h264 decoder optimizations ===
* Quite simply, make our decoder faster
* Qualification task: make our decoder 1% faster
 
 
=== Add support for Bayer RGB colour format ===
Since it is not even clear if this should be implemented in libswscale or libavcodec, this should be discussed before submitting.
There was a related discussion on mplayer-devel once and at least two related FFmpeg issues:
* http://thread.gmane.org/gmane.comp.video.mplayer.devel/48091
* http://roundup.ffmpeg.org/issue1816
* http://roundup.ffmpeg.org/issue1837
* Contact Peter Ross for his latest working patch (Jan-2011)
 
 
=== Finish SoC projects from previous years ===
Some projects are lingering in the dark unfinished. They should be picked up and made ready for inclusion. These projects are potentially less involved than starting from scratch, but also more useful since the probability that the projects get finished should be higher. If some of them are deemed too easy, they could be combined.
 
Unfinished projects from previous years are:
 
2007:
* [[FFmpeg / Libav Summer Of Code#Dirac Encoder and Decoder|Dirac encoder]]
* [[FFmpeg / Libav Summer Of Code#TS Muxer|TS muxer]]
 
2009:
* [[FFmpeg_/_Libav_Summer_Of_Code#Playlist.2FConcatenation_Support_for_FFmpeg|Playlist and concatenation support]]
* [[FFmpeg_/_Libav_Summer_Of_Code#Implement_the_New_Seeking_API_in_Libavformat|New seeking API]]
* [[FFmpeg_/_Libav_Summer_Of_Code#Implementation_of_AVFilter_infrastructure_and_various_audio_filters|AVFilter audio infrastructure]]
 
For the current status of all SoC projects up to date, see [[FFmpeg / Libav Summer Of Code]].
 
=== Implement a Windows Video Presentation 2 decoder ===
* Fourcc: [[WVP2]]
* Samples: http://samples.libav.org/V-codecs/WVP2/
* Suggested qualification task: [[Reverse_Engineering|Reverse engineer]] the frame header parsing or the actual frame data parsing
 
=== RALF Realaudio Lossless ===
* RE and implement a decoder for this format
* Reuse as much libavcodec code as possible
 
=== libvo ===
* Port MPlayer's libvo to ffplay
* Note that this does not just mean to produce a working hack so that ffplay can use xv, but a clean and acceptable wrapper for (most of) libvo.
* An alternative might be to only port the OpenGL part of libvo into a new glplay. This should be discussed before submission.
 
=== Speex Decoder ===
* Specification:  http://speex.org/docs/
* Samples: http://samples.libav.org/A-codecs/speex/
Also see [[Speex]].
 
=== CELT/Opus Decoder ===
* Specification:  http://celt-codec.org/docs
Also see [[CELT]].
 
=== AMR-NB Encoder ===
* Specification: http://www.3gpp.org/ftp/Specs/html-info/26-series.htm
* Samples: http://samples.libav.org/A-codecs/amr/
Also see [[AMR]].


=== VP6 Encoder ===
''Mentor: [[User:lu_zero|Luca Barbato]]''
* Specification: [[On2 VP6]]
* Samples: http://samples.libav.org/V-codecs/VP6/


=== WMV3 Encoder ===
=== Rewrite avserver ===
* Clearly defined task
* Primary goal: Encode video sequences such that they can be decoded by a Windows Media player.


This could either be done by improving [[Interesting Patches#WMV3 encoder by Denis Fortin|this patch]] or by writing the encoder from scratch.
Libav needs a better system to serve streams, the current codebase had a number of design defect showing its age. The new avserver should be written from scratch, leveraging the knowledge piled up.


=== Improve subtitle support ===
The implementation will be incrementally complex and possibly modular.


* Add text-to-bitmap conversion functions
* Implement support to rtsp record/announce
* One with hard-coded bitmaps for characters
* Expose RTMP and RTSP specific API (so the server won't have to use private calls)
* One that utilizes freetype
* Write a simple rtsp, http, rtmp redirector (listen for publish/announce and rebroadcast the received streams)
* Function used will be chosen upon compilation
* Add the capability to serve on-demand content reading from a single path


Adjust existing subtitle support to new ABI
Ideally the first implementation can be made using a poll/event loop and then moved to use threads.
* ABI change: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-January/058521.html


=== Improve Ratecontrol ===
''Mentor: [[User:lu_zero|Luca Barbato]]''
*Primary goal 1: Fast heuristic VBV compliant per macroblock ratecontrol which has a better PSNR/bitrate and better subjective quality/bitrate than the current code.
*Primary goal 2: VBV compliant, rate distortion optimal per macroblock ratecontrol using the viterbi algorithm.
*Secondary goal 1: Fast heuristic scene change detection which detects scene changes more accurately, has better PSNR/bitrate and subjective quality/bitrate than the current heuristic.
*Secondary goal 2: Rate distortion optimal (for the current picture) scene change detection.
*Secondary goal 3: B frames decision which is faster and or has a higher PSNR/bitrate and subjective quality/bitrate than the current code.


=== SILK decoder and/or encoder ===
=== Swscale Cleanup and Additions ===
* A specification was published: http://tools.ietf.org/html/draft-vos-silk-01


Libswscale is an open source library for colourspace conversion. However, it requires significant cleanup to allow
it to support modern features. Knowledge of x86 assembly is preferred, though it can be learnt as part of GSOC.


* Support for more interleaved colourspaces. This is useful because interleaved chroma provides improved cache coherency and would form the basis for future optimisations in libav's decoders.
* Support for the vast number of packed pixel formats (e.g r210, v210). Many are currently implemented as decoders but it is agreed that all uses of such formats are effectively colourspace conversions, they should be included in swscale. This task will involve collaborating with the student of "Adobe DNG Decoder" (if applicable) to rewrite the pixel format headers to support more complex pixel formats.
* Inline asm to yasm conversion
* RGB work (todo: expand this)
* Bayer layouts
* Other general cleanup


=== ATRAC3plus ===
''Mentor: Kieran Kunhya (kierank)''
* Implement a decoder for [[ATRAC3plus]].


=== MPEG-4 SLS ===
== 2nd Tier Project Proposals ==
* libavcodec has currently no support for [[MPEG-4 SLS]]


=== DirectShow capture ===
=== DTS-LBR decoder ===
* Implement an indev for DirectShow capture on Windows (and support A/V synchronisation).
* Qualification: Get [http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/106245 GDI] and [http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/105810 waveform] capture integrated into master with A/V synchronisation. Both patches are almost finished.


=== AAC-LD ===
This is the DTS equivalent to E-AC3 but not technically related to DTS/DTS-HD. It is found in the following specification: http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). The task is to find a way of making the official
* Implement an AAC-LD decoder.
decoder decode just the LBR stream (and not mix it into the main audio) and use that to verify decoder compliance of the decoder you wrote.
The spec may be incomplete or require parts to be reverse engineered from the binary.


=== H.264 MVC ===
''Mentor: ???''
* Implement an H.264 MVC decoder. This format is used in 3D Blu-Rays.


=== SSE2 swscale ===
* swscale largely implements MMX. Add SSE2 support using the yasm macro library.
* Difficult variants: add OpenCL, high bit-depth support, gamma-accurate conversion, better dithering, or edge-directed interpolation.


=== Forward Error Correction ===
[[Category:Libav]]
Add FEC to libavutil - this is relevant to ESA summer of code.
* RTP FEC, standardised as SMPTE 2022M (Possibly the same as Pro-MPEG FEC) - (not 100% sure if this is for all types of RTP or just some)
* MPEG-TS 16 bytes extra - (not sure where this is standardised - in DVB perhaps?)

Latest revision as of 16:17, 13 February 2013

How it works

Google's Summer of Code program is simple: you (the student) work on a project, full-time, during the whole summer, and you get assistance (advice, mentoring) from a seasoned Libav developer who knows his way around the project and has considerable standing in the community. By doing so, you'll learn to operate in an opensource project, you'll get relevant coding experience, and you'll have a chance at earning money while doing fun stuff during the summer. So, you need a project, a mentor, do a qualification task (see below) so we can quickly assess how good a candidate we feel you'll be for the program, and then you can apply.

Selecting a project

Below, you'll find two lists of projects:

  • Projects with a mentor
  • Projects without a mentor

If you choose a project with a mentor, talk to that mentor (see below) and select a suitable qualification task. Once completed, you're eligible for participating in our Summer of Code program. If you choose a project without a mentor, your first job is to find a mentor (see below). Then, once you've found a mentor, continue as before. If you don't like any of the projects, you're free to define your own project and find a mentor as mentioned before (see below for caveats).

Contacting developers/mentors

Once you've found a project (with or without mentor), start talking to the developers of the Libav project. We can often be found on IRC, and you can talk to us on mailinglists also. Hop on irc.freenode.net channel #libav-devel, or talk to us on libav-devel@libav.org. Here, you'll be able to ask around for mentors for projects without a mentor if you need to. If you're trying to define your own project, explain (with reasonable amount of detail) what you intend to achieve and why you think your project should be in our Summer of Code program. Once you've found a mentor, you're good to start your qualification task.

Note that the self-selected mentor needs to have considerable standing in the community to be eligible for mentoring. Likewise, if you choose to define your own Summer of Code project, some community members of considerable standing need to vouch for your project.

Your qualification task

The goal of a qualification task is to see if the mentor and student feel that, together, they will be able to finish the project of their choice. More specifically, the mentor will want to test whether the student has the skills and work ethics to complete a large coding project in a limited amount of time. The student will want to make sure that the mentor provides (useful) assistance when necessary. Therefore, students should select a mentor and a Summer of Code project before starting their work on a qualification task. The qualification task is often (but not necessarily) related to the selected project. For example, if your project will be to reverse engineer a new video codec, the qualification task may be to write a partial bitstream parser for that codec. If your project is to write a muxer for a container format, the qualification task may be to write the code to write the stream header.

There will be a second qualification task for every student: Pick a file of moderate size and reformat it in proper K&R style. The goal of this task is twofold: First it familiarizes students with the style that they will have to write their code in, second it demonstrates that students are able to submit patches from git and go through our review process.

Applying

While you're working on your qualification task, apply at http://www.google-melange.com/. The degree (and detail) with which you've finished your qualification task will determine how likely your project is to be selected. In the past few years, students that completely finished their qualification task always got selected as Summer of Code students, but that may vary depending on the number of spots we get assigned by Google, and the number of students that apply.

1st Tier Project Proposals

1st tier project proposals are project ideas that are reasonably well defined AND have a mentor volunteered.

Assembly Unit Testing Framework

  • Libav has a lot of assembly and not enough tests for it. Your job is to write a unit testing framework for assembly.
  • The framework should work across all supported architectures and operating systems.
  • The framework should measure exactly how fast an individual function is (e.g. using START/STOP_TIMER).
  • The framework should be able to test functions in isolation.
  • x264's checkasm can be used as a reference.
  • The qualification task will be to implement at least one unit test and have an idea of how to do the rest.

Mentor: Daniel Kang (Jumpyshoes on #libav-devel@chat.freenode.net; daniel.d.kang@gmail.com -- ping me on IRC and email me).

HEVC/H265 decoder

  • HEVC is the next-generation video standard from the MPEG standards committee. Your job is to write a decoder that can playback HEVC files.
  • It does not need to be SIMD-optimized, but preferably has relevant functions separated in DSP contexts so they're easy to optimize later on.
  • This task will depend on the current standardisation status of HEVC. Final publication is in 2013 but there may be parts which will not change much anyway.
  • Draft spec: http://www.itscj.ipsj.or.jp/sc29/open/29view/29n12612c.htm
  • HM (draft reference decoder) git mirror: http://hevc.kw.bbc.co.uk/git/w/jctvc-hm.git

Note that this is a difficult project. Do not try to apply unless you already have experience with H.264 as well as reading and writing complex C code.

Mentor: Ronald S. Bultje

DTS-HD decoder

(1) Add support for mixed Core + DTS-HD stream structure
    (DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main
    and commentary tracks.
(2) Add support for XXCh extension (6.1 and 7.1 channels).
(3) Add support for X96 extension (96khz).
(4) Add support for XLL extension (lossless).
(5) Add support for a pure DTS-HD stream structure
    (DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks.
(6) Add support for XBR extension (extra bitrate).


Mentor: Benjamin Larsson

MPEG-4 ALS Roundup

This task is to update and enhance the existing ALS decoder as well as integrate and enhance the rudimentary encoder found at: https://github.com/justinruggles/FFmpeg-alsenc

Possible features are:

  • implement rls-lms in the decoder
  • do correct channel layout/sort handling in the decoder
  • update to current master
  • use codec private options
  • implement encode2(), setting pts and duration
  • document options and examples in encoders.texi
  • come up with a good set of encoding tests for FATE
  • implement mcc/channel sort in the encoder
  • implement rls-lms in the encoder
  • implement float support

Mentor: Justin Ruggles


Opus Decoder

Implement an independent Opus decoder using the publicly-available specification at: http://tools.ietf.org/html/draft-ietf-codec-opus-11

  • The reference source code should only be used as a normative document reference when required (i.e. this should not just be a port of libopus)
  • Fully support Ogg/Opus mapping: https://wiki.xiph.org/OggOpus
  • Handle CELT, SILK, and Hybrid modes (including transitions)
  • (optional) Handle more than 2 channels

Mentor: Justin Ruggles


Adobe DNG Decoder (Basic Support)

Adobe Digital Negative (DNG) is an attempt at a universal file format for raw camera images. Most camera manufacturers have their own proprietary raw image format. Adobe provides tools for converting these to DNG with minimal or no loss of information for more reliable long-term support in a format with an open specification.

The project goal would be to add features required for basic support of DNG files. Some of these include:

  • test/improve TIFF and LJPEG 16-bit decoding support
  • implement both variants of JPEG-in-TIFF in the TIFF decoder
  • add basic handling for Bayer CFA pixel format(s), including demosaicing
  • conversion from camera colorspace to RGB
  • export of DNG/TIFF/Exif metadata

Resources:

Mentor: Justin Ruggles


On2 VP7 decoder

VP7 is a DCT-based video codec. At the moment, it seems to lack a reference implementation, but we do have a spec and may take hint from libavcodec's VP8 and VP6 decoder and libvpx. MPlayer can decode VP7 by loading a binary. To begin working on the project, one has to setup a reference decoder against which to compare our output. This can be done by either writing a wrapper for the binary or (maybe) hacking libvpx itself.

You might want to discuss with us how and where to start. Drop by on IRC if you need help. It is not as difficult as it sounds.

Mentor: Mashiat Sarker Shakkhar


RTMP[E|S|T|TE] protocol implementation

Currently librtmp is required for RTMPE, RTMPS, RTMPT, RTMPTE. The goal of this project is to implement these protocol variants natively in libavformat.


Mentor: Martin Storsjö, Luca Barbato

spin off build system into a separate project

Our build system is neat enough to make into a more general solution to be reused by other projects.

The goal of this project is to achieve exactly that. Intermediate steps will be reading, understanding and documenting the current build system, refactoring parts that can be generalized further and finally making a prototype implementation for libpostproc.

You will require skills in POSIX shell, GNU Make and a firm command of English.

Mentor: Diego Biurrun

Hardware Acceleration API

Libav has two different and incomplete api to provide hardware acceleration wrapping, the project should provide a better abstraction and migrate the currently implemented ones and additional provide more.

  • Draft the API (that will require knowledge of libavcodec)
  • Port vaapi/hwaccel to the new API.
  • Port vdpau to the new API
  • Implement Freescale VPU support
  • Implement TI dce support

Mentor: Luca Barbato

Rewrite avserver

Libav needs a better system to serve streams, the current codebase had a number of design defect showing its age. The new avserver should be written from scratch, leveraging the knowledge piled up.

The implementation will be incrementally complex and possibly modular.

  • Implement support to rtsp record/announce
  • Expose RTMP and RTSP specific API (so the server won't have to use private calls)
  • Write a simple rtsp, http, rtmp redirector (listen for publish/announce and rebroadcast the received streams)
  • Add the capability to serve on-demand content reading from a single path

Ideally the first implementation can be made using a poll/event loop and then moved to use threads.

Mentor: Luca Barbato

Swscale Cleanup and Additions

Libswscale is an open source library for colourspace conversion. However, it requires significant cleanup to allow it to support modern features. Knowledge of x86 assembly is preferred, though it can be learnt as part of GSOC.

  • Support for more interleaved colourspaces. This is useful because interleaved chroma provides improved cache coherency and would form the basis for future optimisations in libav's decoders.
  • Support for the vast number of packed pixel formats (e.g r210, v210). Many are currently implemented as decoders but it is agreed that all uses of such formats are effectively colourspace conversions, they should be included in swscale. This task will involve collaborating with the student of "Adobe DNG Decoder" (if applicable) to rewrite the pixel format headers to support more complex pixel formats.
  • Inline asm to yasm conversion
  • RGB work (todo: expand this)
  • Bayer layouts
  • Other general cleanup

Mentor: Kieran Kunhya (kierank)

2nd Tier Project Proposals

DTS-LBR decoder

This is the DTS equivalent to E-AC3 but not technically related to DTS/DTS-HD. It is found in the following specification: http://www.etsi.org/deliver/etsi_ts/102100_102199/102114/01.03.01_60/ts_102114v010301p.pdf). The task is to find a way of making the official decoder decode just the LBR stream (and not mix it into the main audio) and use that to verify decoder compliance of the decoder you wrote. The spec may be incomplete or require parts to be reverse engineered from the binary.

Mentor: ???