FFmpeg Summer of Code 2012

From MultimediaWiki
Jump to navigation Jump to search

FFmpeg Summer of Code Proposals and Qualification Tasks.

Timeline

March 12th-16th: Project application evaluation.

NOTE: FFmpeg has not been accepted this year, so unless you want to do any of these, the small tasks or qualification tasks just for fun...

Contacting Developers

Find us on irc, server: irc.freenode.net channel: #ffmpeg-devel or contact us by subscribing to the mailing list https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-devel/

Qualification Tasks

To be eligible for a Summer of Code project, we ask you to do a small programming task to prove you know the basics. FFmpeg is a large, complicated collection of code and it's not easy for beginners. There are some ideas for tasks on the Small FFmpeg Tasks page.

1st Tier Project Proposals

These are proposals with a mentor attached.

Baptiste has also offered to mentor.

hwaccel: global architecture

Revisit my older v2 proposal to completely get rid of HW pixel formats. The advantages were: this simplifies things, clean ups code, allows to move more code to the codec library (FFmpeg), allow for fallbacks to SW decoding (at the stream level if HW cannot meet the requirements, not while decoding a particular stream). The key point was to separate identification of the pixel format from the underlying HW accelerator. This will also allow for reading HW surfaces back, when requested if the selected pixel format.

Add encoding and post processing infrastructure.

  • VA-API (hwaccel)
Codecs related:
+ Add support {,M}JPEG decoding
+ Add support for VC-1 interlaced acceleration
+ Add support for H.264 interlaced
Tools related:
Add VA support to ffplay (see my older patches)
 + requires some VO infrastructure work
 + requires hwaccel hooks
 + requires enabling vaapi, dxva, bcm, tidsp, whatever
  • Add VA support to ffmpeg
 + requires VA/DRI API, for no X dependency
 + dependencies: hwaccel v2 to construct pipelines (HW->HW, HW->SW, SW->HW, etc.), or at least allow for VA surface readback wherever necessary.
  • VDPAU (hwaccel)

Update and push my older hwaccel-based VDPAU code. It should still work. :)

Priority: hwaccel v2 infrastructure work first, ffplay enabling second.

Mentor: Gwenole Beauchesne

glplay

Add OpenGL output to ffplay, this should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which you may relicense under the LGPL.

Mentor: Reimar Döffinger

Improve the audio resampling/rematrixing/converting code

  • Right now, we're using libswresample to resample/rematrix audio (samplerate / channels) and to resample the audio format (int, float, 16-bit, 32-bit).
  • Both interleaved and planar audio sample formats are already supported
  • We need SIMD optimization of popular conversions (float-int16, int16-float), (stereo-mono-5.1) and anything else that's frequently used.
  • We need support for alternate conversion functions (e.g. sample format conversion with or without dithering)
  • Fix bugs in current design (none known but there sure are some)

Mentor: Michael Niedermayer

Implement a H.265 / High efficiency video coding (HEVC) decoder

  • Write a basic decoder supporting I, P, and, if time permits, B slices.
  • It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.
  • As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.
  • A draft spec is available at: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/8_San%20Jose/wg11/JCTVC-H1003-v21.zip

Mentor: Michael Niedermayer

H.264 MVC

  • Add MVC support to our H.264 decoder. MVC is used in 3D Blu-Rays.
  • As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.

Mentor: Michael Niedermayer

Libavfilter extension

Libavfilter is the FFmpeg filtering library that started as a 2007 SoC project. It currently supports audio and video filtering and generation support.

The task would consist of writing or porting audio and video filters and eventually fix/extend libavfilter API and design.

In particular the work may focus on porting MPlayer filters which are currently integrated through the mp wrapper. For each port the student should verify that the new filter produces the same output (by comparing the output generated by -vf mp=FILTER and -vf FILTER) and checking that the new integrated filter is not slower.

Prerequisites: good C coding skills, familiarity with git/source code control systems, having some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

For getting more ideas read also the GSoC 2011 libavfilter video proposal and trac libavfilter tickets.

Qualification task: a port or a new implementation of one or more filters.

Mentor: Stefano Sabatini - saste on IRC (possibly with a backup mentor).

Bayer colorspace formats

Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmepg's applicability to RAW still and video photography processing. Tasks:

  • Implement bayer transformations in libswscale (plain C)
  • Add bayer formats to the libavutil pixfmt enumeration routines
  • Extend TIFF decoder to support DNG-Bayer format
  • Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)
  • SIMD optimizations of the libswscale transformations
  • Decoders/specs may be available in the Dcraw project

Qualification task: TBD

Mentor: Peter Ross

Extend image formats support

Improve FFmpeg support for image formats, adding missing formats (e.g. XPM) and extending support for the current ones (e.g. animated GIF, GIF compression, fix PNG todos, add support to animated PNG) etc.

Qualification task: TBD (possibly finally fixing and integrating Måns' zlib decoder that has been unmerged since ages? Or just starting with some small part of the task itself, or implementing format autodetection for imagepipe demuxer)

Mentor: Reimar Döffinger

DTS-HD decoder

(1) Add support for mixed Core + DTS-HD stream structure
    (DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main
    and commentary tracks.
(2) Add support for XXCh extension (6.1 and 7.1 channels).
(3) Add support for X96 extension (96khz).
(4) Add support for XLL extension (lossless).
(5) Add support for a pure DTS-HD stream structure
    (DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks.
(6) Add support for XBR extension (extra bitrate).

Mentor: Benjamin Larsson

Libavdevice API

Currently libavdevice use libavformat API. This is far from being useful for real-time video rendering. Your task is to write new API which resolves this and similar issues and port all already present input and output devices to the new API.

New API should have: 

* Support for output video devices which do not provide own scaler.
* Minimal latency.

Qualification task: write one new video outdev, for example any of, but not limited to: aalib, caca, x11, xv, xover, vesa or gl2 using current libavdevice API.

Mentor: Paul B Mahol

Redesigning the protocol reads

Eliminate active polling while allowing interrupts, and allowing selection on multiple handles.

Mentor: Nicolas George

2nd Tier (need mentor) Project Proposals

Some of the following proposals are also proposed by other organizations, we will try to coordinate this with them so as to avoid duplicate work. We are also happy to hear your personal project ideas ...

AAC decoder improvements

Our AAC decoder does not support low-delay. Part of this task will be to also finish last year's BSAC task. A possible qualification task is to fix a crash in the current BSAC code with one of the samples from the BSAC testing suite.

Sample: https://ffmpeg.org/trac/ffmpeg/ticket/113

AAC encoder improvements

Our AAC encoder does not produce competitive quality per bitrate. Improve the encoder to be better than some other commonly used encoder like libfaac. This requires solid understanding of things like psychoacoustics and rate distortion. A qualification task for this could be to improve the encoder by at least 5% bitrate at the same quality measured by some objective measure.

FF Fuzzer

Write a system like FATE that fuzzes and tests these fuzzed multimedia files under address sanitizer and valgrind with ffmpeg and ffplay. When a crash, or other anomaly is found, it would use git bisect to identify which exact commit introduced the bug. And either display this via some web frontend (similar to fate.ffmpeg.org) or just automatically send an email to some dedicated mailing list. The system has to be robust (there will be infinite loops, OOM conditions and randomly occurring crashes). Its also important that the system is easy to maintain and can filter out duplicates of the same issue.

VC1 interlaced

FFmpeg has code for interlaced VC1, but nearly all samples still do not decode correctly. The task is to finish last year's project. You should be able to find a possible qualification task by testing interlaced samples.



MPEG-4 ALS Roundup

This task is to update and enhance the existing ALS decoder as well as integrate and enhance the rudimentary encoder found at: https://github.com/justinruggles/FFmpeg-alsenc

Possible features are:

  • implement rls-lms in the decoder
  • do correct channel layout/sort handling in the decoder
  • update to current master
  • use codec private options
  • implement encode2(), setting pts and duration
  • document options and examples in encoders.texi
  • come up with a good set of encoding tests for FATE
  • implement mcc/channel sort in the encoder
  • implement rls-lms in the encoder
  • implement float support

Fix and improve FFserver

FFserver has been part of FFmpeg since a long time but due to lack of a motivated maintainer its a bit buggy. For this project you would have to debug and fix many bugs. It requires good skills at reading and understanding other peoples code. As a qualification task you will have to write functioning regression tests for FFserver which implicates some bugfixing to make ffserver produce the same output on all supported platforms.

Support for more subtitle formats

We have libass support now, either a parser (from mplayer) to convert subs into ass or something else.

MKV ordered chapters / playlist support

Get playlist stuff into ffmpeg. Playlist is blocking a few things like quicktime edit list and .asx / .pls files.

Adobe fragmented http in/out

Adobe has a new streaming format.

libavfilter 9/10bit support

Make filters work with higher bitrate codecs/colorspaces

Extend paletted format support

Cleanup framework for handling better with paletted format, write a posterize filter, add support to libswscale palette output (possibly making use of libavcodec/elbg), add support for reading and saving a palette to a file and apply them to the input video (e.g. by creating ad-hoc filters).

Port formats/colorspace support from dcraw or make dcraw wrapper

Dcraw supports many raw camera formats that ffmpeg may not. Port or make a wrapper for this project.

Retrieved from "https://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2012&oldid=14231"