FFmpeg Summer of Code 2013

From MultimediaWiki
Revision as of 03:52, 11 March 2013 by Stefanosa (talk | contribs) (rework initial paragraph, mention timeline explicitly)
Jump to navigation Jump to search

Google Summer of Code is a program that offers students stipends to write code for open source projects. FFmpeg participated to several past editions, and this page is about the Google Summer of Code 2013 edition. In particular, see timeline.

Application period starts on March 18, and March 29 at 19:00 UTC is the application deadline for interested organizations. Please finish up this page before then.

Proposing Ideas

If you have a project idea please contact a developer first at the #ffmpeg-devel IRC channel on Freenode or via the ffmpeg-devel mailing list. A good source of ideas is the FFmpeg bug tracker and FFmpeg Summer of Code 2012 Ideas List.

When adding an idea follow this template for consistency:

== Task Title ==

'''Description:''' A few sentences or a short paragraph describing the task.

'''Expected results:''' Bulleted list or paragraph describing what the student is expected to achieve.

'''Prerequisites:''' Skills or knowledge required by student.

'''Mentor:''' List mentor and backup mentor if there is one and contact info such as IRC name or email address.

Mentored tasks

H264 MVC

Description: MVC samples exist, the codec is used on Bluray media, FFmpeg is missing a decoder. Since this task also consists of some changes in the current architecture, it is especially important that this task is discussed on the ffmpeg-devel mailing list. As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.

Mentor: Michael Niedermayer

Animated Portable Network Graphics (APNG)

Description: FFmpeg currently does not support Animated PNGs.

Specification: https://wiki.mozilla.org/APNG_Specification

Animated PNG example bouncing beach ball.png

Expected results:

  • APNG demuxer
    • implement robust probing:
      • PNG images are not misdetected as APNG animations
      • APNG animations are not misdetected as PNG images
    • splits stream into sensible packets (so they can be easily reused in APNG muxer)
    • survives fuzzing (zzuf)
    • add FATE coverage, coverage should be at least 70%
    • test code under valgrind so no invalid reads/writes happen
  • APNG decoder
    • use existing PNG decoder code (write decoder in same file)
    • implement parsing of all APNG chunks (acTL, fcTL, fdAT)
    • error handling
    • survives fuzzing (zzuf)
    • add test for FATE, coverage should be at least 75%
    • CRC checksum validation
    • test code under valgrind so no invalid reads/writes happen
  • APNG muxer && APNG encoder
    • use existing PNG encoder code (write encoder in same file)
    • write compliant files, make sure they play correctly in major web browsers that support APNG
    • add test for FATE

Prerequisites: C coding skills, familiarity with git/source code control systems.

Qualification Task: Implement format autodetection for imagepipe & image demuxer

Mentor: Paul B Mahol

Misc Libavfilter extension

Lavfi-gsoc-filter-vintage-illustration.jpg

Description: Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.

Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.

Some ideas for more filters:

  • a frequency filtering domain filter relying on the FFT utils in libavcodec
  • a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI
  • a lua scripting filter, which allows to implement filtering custom logic in lua

For more ideas check: trac libavfilter tickets.

Expected results: Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.

Prerequisites: C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

Qualification task: write or port one or more filter

Mentor: Stefano Sabatini. Backup mentor: Clément Bœsch (ubitux on IRC).


Three-two pulldown diagram (telecine).png

Inverse telecine filter

Description: Libavfilter has all kind of filters but is really lacking an IVTC one. It's a must-have we have been delaying for too long.

Various very good filters already exists in other projects, notably TIVTC (basically the reference filter), which have been ported/rewritten recently in a portable version (VIVTC). Some other solutions exists in the MPlayer project (something like 4 or 5 filters) but their efficiency is questionable. The VLC project also has its own complete toolkit for inverse telecine which might be interesting to look at, especially for its very complete documentation on the topic.

Expected results: Write/port a complete and efficient inverse telecine filter to libavfilter.

Prerequisites: C coding skills, familiarity with git/source code control systems. Some familiarity with the DVD encoding process and the image processing techniques would be a bonus but are not strictly required.

Qualification task: Make a comparative report about the current filters in various projects (MPlayer, TIVTC, VIVTC, VLC, Handbrake, ...) with all kind of different inputs, notably mixed and badly broken content. This is a very important step that has to be done before writing any code, and will be extremely useful as a future reference.

Mentor: Clément Bœsch (ubitux on IRC). + Derek?


Subtitles

Description: FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carry an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.

Subtitles-sensei.jpg

Expected results:

  • Write the support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray.
  • Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.
  • Improve the API to facilitates a proper integration of subtitles into libavfilter.

Prerequisites: C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.

Qualification task: write one subtitles demuxer or muxer, and do some improvements to the different layers of the subtitles (to make sure the whole chain is understood).

Mentor: Clément Bœsch (ubitux on IRC).

glplay

Opengl logo.jpg

Description: The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.

Mentor: TBD Backup: Reimar Döffinger


Unmentored tasks

AAC

Description: FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.

  • AAC LD decoder
  • AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples
  • AAC SSR decoder
  • AAC 960/120 MDCT window
  • AAC multi-channel encoding

See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.

DTS / DCA

Description: FFmpeg contains a DTS decoder and an experimental DTS encoder. Both are missing some features.

  • DTS-HD decoder improvements: A possible qualification task is to implement ticket #1920
    • Add support for X96 extension (96khz)
    • Add support for XLL extension (lossless)
    • Add support for pure DTS-HD streams that do not contain a DTS core
    • Add support for multiple assets
    • Add support for LBR extension
  • DTS encoder improvements

MPEG-4 ALS encoder

Description: A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on github. For this task, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the reference codec and specifications. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.

Expected results:

Mandatory objectives

  • Update the existing encoder to fit into the current codebase.
  • Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.
  • Push the encoder into the codebase after iterating through the review process on the developing mailing list.

Optional goodies

  • Ensure the FFmpeg decoder processes all generated files without warnings.
  • Enhance the rudimentary feature set of the encoder.

Prerequisites: C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.

Qualification task: Some good idea needed!

Mentor: Mentor needed!


TrueHD encoder

Description: FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.

Opus decoder

Description: Opus decoding is currently supported through the external libopus library

  • Write a native decoder, continue working on the existing unfinished implementation

A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.

HEVC / H265

Description: The specification was finished, FFmpeg needs a decoder for this new format.

  • Write a basic decoder supporting I, P, and, only if time permits, B slices.
  • It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.

As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.

VC-1 interlaced

Description: The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.

  • Implement missing interlace features
  • Make more reference samples bit-exact

As a qualification task, you should try to find a bug in the current decoder implementation and fix it.

JPEG 2000

Jpeg2000.jpg

Description: FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.


VP7

Description: Not many VP7 samples are in the wild, but no open-source decoder exists although a specification exists. Write a decoder that reuses as much as possible of existing FFmpeg code, it is likely that functions of the existing decoders for On2-based formats will be useful.