FFmpeg / Libav Summer Of Code 2011

From MultimediaWiki
Jump to navigation Jump to search

Qualification tasks

For us to consider your application for SoC we require a completed qualification task. Many Summer-of-Code projects (in the list below) have specific qualification tasks. These tasks are meant to make you familiar with the code that you will be working with, are at approximately the same difficulty level as the actual Summer-of-Code project itself (just a lot smaller), and often already provide you with a jumpstart into your Summer-of-Code project. We suggest the following order of events:

  • First, select a Summer-of-Code project (either from the list below, but in some cases you may also come up with your own)
  • Second, discuss this project with the person that will mentor it. If a mentor is listed, talk to him on IRC, via email or so. If no mentor is listed, find one by emailing the FFmpeg-soc mailinglist.
  • With your mentor, discuss the most appropriate qualification task for your Summer-of-Code project.

If no specific qualification task is listed for your project of interest, you can discuss with your mentor to choose a task from the Small Tasks list or the Interesting Patches list instead. If your prospective mentor agrees, please send an email to the FFmpeg-soc mailing list to inform that you are working on it (to avoid duplicated work). You can discuss progress on your qualification task and initial review of your finished qualification task on the FFmpeg-soc mailinglist (sometimes the patch will have to go through final review on the main FFmpeg development mailinglist before it can be applied also). The qualification task is considered completed when your patch is accepted to our main SVN tree.

Before posting to the FFmpeg-soc mailinglist, make sure you read and understand our netiquette guidelines, especially avoid top-posting and thread-hijacking (note that if you don't understand one of those terms, make sure to have understood them before writing your first post). Before you send us your patch, read our development guidelines and make sure your patch fulfills all the requirements stated there. You should also familiar with the programs diff, patch and Subversion. You have to learn these basics on your own before you start, we will not teach them to you during the application process.

A completed FFmpeg qualifications task is also accepted as a qualification task for the VLC organization (does NOT include x264).

1st Tier Project Proposals

1st tier project proposals are project ideas that are reasonably well defined AND have a mentor volunteered.

BSAC AAC Decoder

  • Separate AAC Huffman decoding and dequantization from the rest of AAC decoder.
  • Write a BSAC AAC decoder.
  • Reuse as much of the existing AAC decoder and libavcodec as possible.

Mentor: Alex Converse

Young Han Lee has completed a qualification task (AAC LTP profile) for this project

Libav Usage Example, Tutorial and test suite

  • Prepare a serie of example program of increasing complexity
    • Simple remuxer
    • Simple decoding loop
    • Simple encoding loop
    • Simple transcoding loop
    • Simple producer + consumer scenario with the network protocols
    • ...
    • Optionally it would lead to prepare a leaner equivalent to ffmpeg and ffplay
  • Examine when there are inconsistencies with the current API (e.g. unexpected timestamp mangling, packet splitting) and fix the issues.
  • Integrate the example programs as components of FATE.

Mentor: Luca Barbato

VC-1 decoder missing features

  • Add support for interlaced streams as used in Bluray recordings to the VC-1 decoder.
  • Add support for chroma/luma scaling, used in RTP/VC1.
  • Add support for slice decoding, used in RTP/VC1.
  • Add optimizations for x86-32/64 (or ARM), so that HD streams play back faster.
  • This includes fixing some reference streams

Mentor: Kostya Shishkov, Ronald S. Bultje and (maybe) Christophe Gisquet

Shahriman AMS is working on a qualification task for this project

Windows Television (WTV) muxer

  • Qualification task: develop a prototype WTV muxer that can produce files then able to be demuxed by FFmpeg.
  • The objective of this project is produce files that are compatible with Microsoft Windows Media Center. To achieve this, one must determine which of the many undocumented data types are expected by Microsoft Stream Buffer Engine, discover their format, and implement these in the muxer.
  • The muxer must support metadata and indexing.
  • Clearly defined task; reverse engineering skills required.

Mentor: Peter Ross

Zhentan Feng is working on the qualification task for this project

DTS Encoder

  • Specification: http://wiki.multimedia.cx/index.php?title=Mirror
  • Sample files: http://samples.mplayerhq.hu/A-codecs/DTS/
  • Qualification: Extend the encoder in the SoC tree (port the float transform from the decoder, fix the LFE channel generation).
  • Primary goal: Encoder that can produce multi sample rate, multi channel files and multi bit-rate. Wav and raw muxing support.
  • Secondary goal: Techniques from specification implemented, optimal codebook usage, vector quantization, simple psycho acoustics.
  • Third goal: Advanced psycho acoustics.

Mentor: Benjamin Larsson

Xiang Wang is working on a qualification task for this project

Hardware accelerated decoder for Android devices

Most Android phones supports decoding H264 via an OpenMAX decoder interface. This interface is exposed through the Stagefright multimedia framework. The flashplugin uses this to be able to have hardware accelerated decode of H264 bitstreams from Android user space. As many Android devices will not be upgraded to Android 2.3 or users don't want to root/reflash their firmware this will enable a lot of devices to install a libavcodec based media player while still having hardware accelerated H264 decode.

Mentor: Martin Storsjö, (co-mentor) Benjamin Larsson

Multithreading support

The goal of this task will be to improve the multithreading support in FFmpeg. The task is relatively unspecific, but your basic goal will be to add multithreading support to a variety of decoders in FFmpeg, by using existing infrastructure. Examples goals could include:

  • Help merging the remaining bits of ffmpeg-mt into the main repository (i.e. H.264/MPEG decoder multithreading)
  • Fix problems with FFmpeg's slice-based H.264 decoder threading (ask Jason for details here)
  • Implement functions in the ffmpeg/ffplay binaries to autodetect the number of CPUs/cores and autodetect the number of threads to use (if unspecified)
  • define ways to use both slice-based and frame-based multithreading at the same time if decoders (and files) support both
  • Multithreading in other decoders that currently don't support it
  • implement slice-based and/or frame-based threading in FFmpeg's VP8 decoder
  • implement frame-based threading in MJPEG decoder
  • implement frame-based (or if you want to hurt yourself, try slice-based) multithreading in VC-1 decoder

Mentor: (hopefully) Alexander Strange and Ronald S. Bultje

Libavfilter video work

Libavfilter is the FFmpeg filtering library that started as a 2007 SoC project. It replaced the now removed vhook subsystem. Most of it is already part of the FFmpeg main source tree, but there a few bits remaining. This project would consist in a combination of the following tasks:

  • Get the remaining bits of the SoC tree committed (this includes: the rotate, split, and fps filters)
  • Port all the missing filters from MPlayer (do not forget asking the authors if it is ok to release them under the LGPL)
  • Framework: implement dynamic-reconfiguration of the filterchain, for supporting dynamic size/format changes
  • Port filters from other frameworks (mjpeg-tools, effectv, frei0r, virtualdub, vlc, etc...)
  • Write wrappers for more image processing libraries and filtering frameworks (libgimp, libgraphicsmagic, weed), extend the libopencv wrapper for supporting more filters (this may need implemented float image support in libswscale)
  • Write more filters (possibly starting from already posted filters which for a reason or another were never committed, e.g. concatenate, fish, 2xsai, select, lut, eval, posterize, elbg/posterize etc.)
  • Implement a compose filter (suggestion for a better name?), for creating a mosaic of the input video streams
  • Write a movie sink (e.g. it could be useful for implementing a display functionality, e.g. when the video is dumped to an output device)
  • Write a frequency domain filter using the FFT implementation in libavcodec
  • Write a Matlab/Octave/SAGE scripting wrapper (assuming it can be done)
  • More ideas?

Mentor: Stefano Sabatini (possibly with a backup mentor)

Libavfilter audio work

At the moment, FFmpeg filtering library has incomplete support for handling audio. This task would consist of a combination of the following tasks:

  • Fix the remaining outstanding issues and complete the audio framework integration (check saste's audio-filters FFmpeg branch on gitorious)
  • Make the resampling filter works for several combinations of sample formats and channels
  • Write a visualization filter as proof-of-concept of a filter that works with both video and audio
  • Extend/improve libav* audio resampling/rematrixing/requantization capabilities (this may need to re-design the audio API, maybe it deserves a task on its own)
  • Write/port basic audio filters (check for example MPlayer/libaf)
  • Write wrapper for audio frameworks (sox, ladspa, ...)

Mentor: Stefano Sabatini (possibly with a backup mentor)

Mina Nagy Zaki is working on a qualification task for this project

Flash screen video (1)/2

  • Write a decoder for the screen video 2 format
  • Enhance the encoder patch posted to ffmpeg-devel so it can be committed
  • Write optimal 2-pass encoding support for the flash screen video 1 and possibly 2 encoder (djinkstra graph)

Mentor: Benjamin Larsson (possibly some one else)

Rewrite the audio format conversion code

  • right now, we're using audioresample to resample audio (samplerate / channels) and audioconvert to resample audio format (int, float, 16-bit, 32-bit). We'd like a new, swscale-style implementation that combines these two in a single codebase
  • design should allow for both interleaved and planar audio sample formats
  • design should allow for SIMD optimization of popular conversions (float-int16, int16-float)
  • design should allow for alternate conversion functions (e.g. sample format conversion with or without dithering and different sample rate conversion algorithms)
  • fix bugs in current design (clipping, overflows when going from float to int, questionable rounding)

Mentor: Justin Ruggles

MXF support

  • Support Clip Wrapping, Op1B, Avid proprietary files
  • Support muxing DV, H.264
  • Seeking support using index

Mentor: Baptiste Coudurier

Standards Compliant TS Muxer

  • This is harder than it might seem at first because of the myriad of codecs and muxing rules.
  • In order to complete this task you must find (or ask for) the spec, read it and understand the buffering models and what you will need to do for each codec. It's much easier to start with MPEG codecs to begin with. In most cases this will involve extracting or guessing information from the input stream.
  • Sometimes spec-incompliant modes might be ok such as H.264 in TS files without HRD for iPod streaming.
  • Later work can focus on remuxing

Mentor: Kieran Kunhya

1st tier proposals of last year

These are the same as above-mentioned 1st tier proposals, but we're not sure yet if we'll repeat them. Once mentors are OK, please move them up.

NEW Seeking API

  • primary goal: implement a new seeking API in libavformat
    • implement av_seek_file in libavformat
    • implement compatible new seek_file for all AVInputFormat porting existing seek function if possible.
    • implement av_build_index function which will build an AVIndex for the file
    • implement av_export_index function which will save AVIndex in a file which can be loaded later.

Mentor: Baptiste Coudurier

AACS implementation

  • this task probably is to get libbluray integrated with vlc/mplayer/ffmpeg and working with as many discs as possible.
  • Add the ability to encode and decode using Advanced Access Content System to FFmpeg.
  • Specifications: http://www.aacsla.com/specifications/
  • existing implementation e.g. DumpHD: http://forum.doom9.org/showthread.php?t=123111
  • Most parts (BD-J, MKB, title key generation) probably do not belong into FFmpeg, this should be discussed with us before submitting an application
    • possible solution: only implement "lowest" level (decode given the correct title key) but implement CSS en- and decryption as secondary goal

Mentor: Reimar Döffinger

h264 decoder optimizations

  • Quite simply, make our decoder faster
  • Qualification task: make our decoder 1% faster

Mentor: Michael Niedermayer

jlindgren is interested in this task

Add support for Bayer RGB colour format

Since it is not even clear if this should be implemented in libswscale or libavcodec, this should be discussed on ffmpeg-devel or ffmpeg-soc before submitting. There was a related discussion on mplayer-devel once and at least two related FFmpeg issues:

Mentor: Michael Niedermayer

2nd Tier Project Proposals

All that separates these proposals from their 1st tier brethren is a lack of mentor.

Finish SoC projects from previous years

Some projects are lingering in the dark unfinished. They should be picked up and made ready for inclusion. These projects are potentially less involved than starting from scratch, but also more useful for FFmpeg since the probability that the projects get finished should be higher. If some of them are deemed too easy, they could be combined.

Unfinished projects from previous years are:

2007:

2008:

2009:

For the current status of all SoC projects up to date, see FFmpeg Summer Of Code.

rukhsana afroz is working on jpeg2000 decoder

Implement a Windows Video Presentation 2 decoder

RALF Realaudio Lossless

  • RE and implement a decoder for this format
  • Reuse as much libavcodec code as possible

libvo

  • Port MPlayer's libvo to ffplay
  • Note that this does not just mean to produce a working hack so that ffplay can use xv, but a clean and acceptable wrapper for (most of) libvo.
  • An alternative might be to only port the OpenGL part of libvo into a new glplay. This should be discussed on ffmpeg-devel before submission.

Speex Decoder

Also see Speex.

CELT/Opus Decoder

Also see CELT.

AMR-NB Encoder

Also see AMR.

VP6 Encoder

WMV3 Encoder

  • Clearly defined task
  • Primary goal: Encode video sequences such that they can be decoded by a Windows Media player.

This could either be done by improving this patch or by writing the encoder from scratch.

Improve subtitle support

  • Add text-to-bitmap conversion functions
  • One with hard-coded bitmaps for characters
  • One that utilizes freetype
  • Function used will be chosen upon compilation

Adjust existing subtitle support to new ABI

Improve Ratecontrol

  • Primary goal 1: Fast heuristic VBV compliant per macroblock ratecontrol which has a better PSNR/bitrate and better subjective quality/bitrate than the current code.
  • Primary goal 2: VBV compliant, rate distortion optimal per macroblock ratecontrol using the viterbi algorithm.
  • Secondary goal 1: Fast heuristic scene change detection which detects scene changes more accurately, has better PSNR/bitrate and subjective quality/bitrate than the current heuristic.
  • Secondary goal 2: Rate distortion optimal (for the current picture) scene change detection.
  • Secondary goal 3: B frames decision which is faster and or has a higher PSNR/bitrate and subjective quality/bitrate than the current code.

SILK decoder and/or encoder

DTS-HD Master Audio decoder

  • You should know exactly what you do if you submit an application.

ATRAC3plus

MPEG-4 SLS

DirectShow capture

  • Implement an indev for DirectShow capture on Windows (and support A/V synchronisation).
  • Qualification: Get GDI and waveform capture integrated into FFmpeg SVN with A/V synchronisation. Both patches are almost finished.

AAC-LD

  • Implement an AAC-LD decoder.

H.264 MVC

  • Implement an H.264 MVC decoder. This format is used in 3D Blu-Rays.

H.264 4:2:2 (and 4:4:4?)

  • Implement a 4:2:2 (and 4:4:4?) H.264 decoder.

SSE2 swscale

  • swscale largely implements MMX. Add SSE2 support using the yasm macro library.
  • Difficult variants: add OpenCL, high bit-depth support, gamma-accurate conversion, better dithering, or edge-directed interpolation.

Forward Error Correction

Add FEC to libavutil - this is relevant to ESA summer of code.

  • RTP FEC, standardised as SMPTE 2022M (Possibly the same as Pro-MPEG FEC) - (not 100% sure if this is for all types of RTP or just some)
  • MPEG-TS 16 bytes extra - (not sure where this is standardised - in DVB perhaps?)
Retrieved from "https://wiki.multimedia.cx/index.php?title=FFmpeg_/_Libav_Summer_Of_Code_2011&oldid=13529"