FFmpeg Summer of Code 2012

From MultimediaWiki
Revision as of 00:38, 12 March 2012 by Michael (talk | contribs) (moved to 1st, so remove)
Jump to navigation Jump to search

FFmpeg Summer of Code Proposals and Qualification Tasks.

Timeline

March 12th-16th: Project application evaluation.

Contacting Devels

Find us on irc, server: irc.freenode.net channel: #ffmpeg-devel or contact us by subscribing to the mailing list https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel/

Qualification Tasks

To be eligible for a Summer of Code project, we ask you to do a small programming task to prove you know the basics. FFmpeg is a large, complicated collection of code and its not easy for beginners. There are some ideas for tasks on the Small FFmpeg Tasks page.

1st Tier Project Proposals

These are proposals with a mentor attached.

Baptiste has also offered to mentor.

glplay

Add OpenGL output to ffplay, this should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which you may relicense under the LGPL.

Mentor: Reimar Döffinger

Improve the audio resampling/rematrixing/converting code

  • right now, we're using libswresample to resample/rematrix audio (samplerate / channels) and to resample the audio format (int, float, 16-bit, 32-bit).
  • both interleaved and planar audio sample formats are already supported
  • We need SIMD optimization of popular conversions (float-int16, int16-float), (stereo-mono-5.1) and anything else thats frequently used.
  • We need support for alternate conversion functions (e.g. sample format conversion with or without dithering)
  • fix bugs in current design (none known but there sure are some)

Mentor: Michael Niedermayer

Implement a H.265 / High efficiency video coding (HEVC) decoder

  • Write a basic decoder supporting I P and if time permits B slices.
  • It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.
  • As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, its not something for the average SOC student.
  • A draft spec is available at: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/8_San%20Jose/wg11/JCTVC-H1003-v21.zip

Mentor: Michael Niedermayer

H.264 MVC

  • Add MVC support to our H.264 decoder. MVC is used in 3D Blu-Rays.
  • As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.

Mentor: Michael Niedermayer

Libavfilter extension

Libavfilter is the FFmpeg filtering library that started as a 2007 SoC project. It currently supports audio and video filtering and generation support.

The task would consist into writing or porting audio and video filters and eventually fix/extend libavfilter API and design.

In particular the work may focus on porting MPlayer filters which are currently integrated through the mp wrapper. For each port the student should verify that the new filter produces the same output (by comparing the output generated by -vf mp=FILTER and -vf FILTER) and checking that the new integrated filter is not slower.

Prerequisites: good C coding skills, familiarity with git/source code control systems, having some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

For getting more ideas read also the GSoC 2011 libavfilter video proposal and trac libavfilter tickets.

Qualification task: a port or a new implementation of one or more filters.

Mentor: Stefano Sabatini - saste on IRC (possibly with a backup mentor).

Bayer colorspace formats

Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaded FFmepg's applicability to RAW still and video photography processing. Tasks:

  • Implement bayer transformations in libswscale (plain C)
  • Add bayer formats to the libavutil pixfmt enumeration routines
  • Extend TIFF decoder to support DNG-Bayer format
  • Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)
  • SIMD optimisations of the libswscale transformations
  • decoders/specs may be available in the Dcraw project

Qualification task: TBD

Mentor: Peter Ross

Extend image formats support

Improve FFmpeg support for image formats, adding missing formats (e.g. XPM) and extending support for the current ones (e.g. animated GIF, GIF compression, fix PNG todos, add support to animated PNG) etc.

Qualification task: TBD (possibly finally fixing and integrating Måns' zlib decoder that has been unmerged since ages? Or just starting with some small part of the task itself, or implementing format autodetection for imagepipe demuxer)

Mentor: Reimar Döffinger

2nd Tier (need mentor) Project Proposals

Some of the following proposals are also proposed by other organizations, we will try to coordinate this with them so as to avoid duplicate work. We are also happy to hear your personal project ideas ...

AAC decoder improvments

Our AAC decoder does not support low-delay. Part of this task will be to also finish last year's BSAC task. A possible qualification task is to fix a crash in the current BSAC code with one of the samples from the BSAC testing suite.

AAC encoder improvments

Our AAC encoder does not produce competetive quality per bitrate. Improve the encoder to be better than some other commonly used encoder like libfaac. This requires solid understanding of things like psychoacoustics and rate distortion. A qualification task for this could be to improve the encoder by at least 5% bitrate at the same quality meassured by some objective measure.

FF Fuzzer

Write a system like FATE that fuzzes and tests these fuzzed multimedia files under address sanitizer and valgrind with ffmpeg and ffplay. When a crash, or other anomaly is found, it would use git bisect to identify which exact commit introduced the bug. And either display this via some web frontend (similar to fate.ffmpeg.org) or just automatically send an email to some dedicated mailing list. The system has to be robust (there will be infinite loops, OOM conditions and randomly occuring crashes). Its also important that the system is easy to maintain and can filter out duplicates of the same issue.

VC1 interlaced

FFmpeg has code for interlaced VC1, but nearly all samples still do not decode correctly. The task is to finish last year's project. You should be able to find a possible qualification task by testing interlaced samples.


DTS-HD decoder

(1) Add support for mixed Core + DTS-HD stream structure
    (DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main
    and commentary tracks.
(2) Add support for XXCh extension (6.1 and 7.1 channels).
(3) Add support for X96 extension (96khz).
(4) Add support for XLL extension (lossless).
(5) Add support for a pure DTS-HD stream structure
    (DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks.
(6) Add support for XBR extension (extra bitrate).

MPEG-4 ALS Roundup

This task is to update and enhance the existing ALS decoder as well as integrate and enhance the rudimentary encoder found at: https://github.com/justinruggles/FFmpeg-alsenc

Possible features are:

  • implement rls-lms in the decoder
  • do correct channel layout/sort handling in the decoder
  • update to current master
  • use codec private options
  • implement encode2(), setting pts and duration
  • document options and examples in encoders.texi
  • come up with a good set of encoding tests for FATE
  • implement mcc/channel sort in the encoder
  • implement rls-lms in the encoder
  • implement float support

Fix and improve FFserver

FFserver has been part of FFmpeg since a long time but due to lack of a motivated maintainer its a bit buggy. For this project you would have to debug and fix many bugs. It requires good skills at reading and understanding other peoples code. As a qualification task you will have to write functioning regression tests for FFserver which implicates some bugfixing to make ffserver produce the same output on all supported platforms.

Reverse engineer TAK codec and write decoder for it

TAK format is partially already documented. You need to revisit that documentation and update and/or fix any missing/wrong inforrmation. Reverse engineer codec and write working bitexact decoder.

Support for more subtitle formats

we have libass support now, either a parser (from mplayer) to convert subs into ass or something else.

MKV ordered chapters / playlist support

get playlist stuff into ffmpeg. playlist is blocking a few things like quicktime edit list and .asx / .pls files.

Adobe fragmented http in/out

adobe has a new streaming format.

libavfilter 9/10bit support

make filters work with higher bitrate codecs/colorspaces

Fix copying video between formats

lots of h264 streams in flv, mp4, mkv, mpegts. people wish to remux these into various formats so it works on their hardware (ps3, ipod etc).

https://ffmpeg.org/trac/ffmpeg/ticket/796
https://ffmpeg.org/trac/ffmpeg/ticket/822
https://ffmpeg.org/trac/ffmpeg/ticket/954
https://ffmpeg.org/trac/ffmpeg/ticket/976

Extend paletted format support

Cleanup framework for handling better with paletted format, write a posterize filter, add support to libswscale palette output (possibly making use of libavcodec/elbg), add support for reading and saving a palette to a file and apply them to the input video (e.g. by creating ad-hoc filters).

Port formats/colorspace support from dcraw or make dcraw wrapper

Dcraw supports many raw camera formats that ffmpeg may not. port or make a wrapper for this project.