Libav Summer Of Code 2013: Difference between revisions

From MultimediaWiki
Jump to navigation Jump to search
 
(4 intermediate revisions by 3 users not shown)
Line 84: Line 84:


'''Mentor: [[User:DonDiego|Diego Biurrun]]'''
'''Mentor: [[User:DonDiego|Diego Biurrun]]'''
'''Co-Mentor: [[User:lu_zero|Luca Barbato]]'''


=== The Grand Refactoring (working title) ===
=== The Grand Refactoring (working title) ===


Many parts of the libav codebase are still unnecessarily monolithic. This makes custom builds that only support a subset of the multitude of formats we have larger than they need to be. The goal of this project will be to locate parts that can be separated and refactor the code so that each subpart can be compiled standalone so as to not increase the size of a custom configuration without the part.
Many parts of the libav codebase are still unnecessarily monolithic. This makes custom builds that only support a subset of the multitude of formats we have larger than they need to be and slows down the compiling time on multicore systems.
 
The goal of this project will be to locate parts that can be separated and refactor the code so that each subpart can be compiled standalone so as to not increase the size of a custom configuration without the part.


A suitable qualification task for this project is picking a simple encoder/decoder pair or some part of dsputil (harder) and splitting it cleanly.
A suitable qualification task for this project is picking a simple encoder/decoder pair or some part of dsputil (harder) and splitting it cleanly.


'''Mentor: [[User:DonDiego|Diego Biurrun]]'''
'''Mentor: [[User:DonDiego|Diego Biurrun]]'''
'''Co-Mentor: [[User:lu_zero|Luca Barbato]]'''


=== Rewrite the ASF muxer and demuxer ===
=== Rewrite the ASF muxer and demuxer ===
Line 130: Line 136:


'''Mentor: [[User:elenril|Anton Khirnov]]'''
'''Mentor: [[User:elenril|Anton Khirnov]]'''
'''Co-Mentor: [[User:Jruggle|Justin Ruggles]]'''
=== Adobe DNG Decoder (Basic Support) ===
Adobe Digital Negative (DNG) is an attempt at a universal file format for raw camera images. Most camera manufacturers have their own proprietary raw image format. Adobe provides tools for converting these to DNG with minimal or no loss of information for more reliable long-term support in a format with an open specification.
The project goal would be to add features required for basic support of DNG files. Some of these include:
* test/improve TIFF and LJPEG 16-bit decoding support
* implement both variants of JPEG-in-TIFF in the TIFF decoder
* add basic handling for Bayer CFA pixel format(s), including demosaicing
* conversion from camera colorspace to RGB
* export of DNG/TIFF/Exif metadata
Resources:
* [http://en.wikipedia.org/wiki/Digital_Negative Wikipedia Article]
* [http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf Specification]
* DNG samples can be created from other raw formats using the free [http://www.adobe.com/products/photoshop/extend.displayTab2.html DNG Converter] program
* A good place to find raw camera samples is http://www.imaging-resource.com
'''Mentor: [[User:Jruggle|Justin Ruggles]]'''
=== Streaming protocol improvements ===
* Implement support for producing data for Adobe HTTP Dynamic Streaming and MPEG DASH
* Implement support for receiving the same
Alternatively you can also implement support for another new streaming protocol we currently don't support, or improve the ones we currently support.
'''Mentor: [[User:mstorsjo|Martin Storsjö]]'''
'''Co-Mentor: [[User:lu_zero|Luca Barbato]]'''


== 2nd Tier Project Proposals ==
== 2nd Tier Project Proposals ==
Line 209: Line 247:


The initial SILK code had been written as result of the previous attempt to support Opus
The initial SILK code had been written as result of the previous attempt to support Opus
'''Mentor: [[User:Jruggle|Justin Ruggles]]'''
=== Adobe DNG Decoder (Basic Support) ===
Adobe Digital Negative (DNG) is an attempt at a universal file format for raw camera images. Most camera manufacturers have their own proprietary raw image format. Adobe provides tools for converting these to DNG with minimal or no loss of information for more reliable long-term support in a format with an open specification.
The project goal would be to add features required for basic support of DNG files. Some of these include:
* test/improve TIFF and LJPEG 16-bit decoding support
* implement both variants of JPEG-in-TIFF in the TIFF decoder
* add basic handling for Bayer CFA pixel format(s), including demosaicing
* conversion from camera colorspace to RGB
* export of DNG/TIFF/Exif metadata
Resources:
* [http://en.wikipedia.org/wiki/Digital_Negative Wikipedia Article]
* [http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf Specification]
* DNG samples can be created from other raw formats using the free [http://www.adobe.com/products/photoshop/extend.displayTab2.html DNG Converter] program
* A good place to find raw camera samples is http://www.imaging-resource.com


'''Mentor: [[User:Jruggle|Justin Ruggles]]'''
'''Mentor: [[User:Jruggle|Justin Ruggles]]'''

Latest revision as of 22:13, 29 March 2013

How it works

Google's Summer of Code program is simple: you (the student) work on a project, full-time, during the whole summer, and you get assistance (advice, mentoring) from a seasoned Libav developer who knows the project and has considerable standing in the community. By doing so, you'll learn to operate in an open source project, you'll get relevant coding experience, and you'll have a chance at earning money while doing fun stuff during the summer. So, you need a project, a mentor, a qualification task (see below) so we can quickly assess how good a candidate we feel you'll be for the program, and then you can apply.

Selecting a project

Below, you'll find two lists of projects:

  • Projects with a mentor
  • Projects without a mentor

If you choose a project with a mentor, talk to that mentor (see below) and select a suitable qualification task. Once completed, you're eligible for participating in our Summer of Code program. If you choose a project without a mentor, your first job is to find a mentor (see below). Then, once you've found a mentor, continue as before. If you don't like any of the projects, you're free to define your own project and find a mentor as mentioned before (see below for caveats).

Contacting developers/mentors

Once you've found a project (with or without mentor), start talking to the developers of the Libav project. We can often be found on IRC, and you can talk to us on the mailing lists as well. Hop on irc.freenode.net, channel #libav-devel, or mail us at libav-devel@libav.org. There, you'll be able to ask around for persons to guide you on projects without a mentor if you need to. If you're trying to define your own project, explain (with reasonable amount of detail) what you intend to achieve and why you think your project should be in our Summer of Code program. Once you've found a mentor, you're good to start your qualification task.

Note that the self-selected mentor needs to have considerable standing in the community to be eligible for mentoring. Likewise, if you choose to define your own Summer of Code project, some community members of considerable standing need to vouch for your project.

Your qualification task

The goal of a qualification task is to see if the mentor and student feel that, together, they will be able to finish the project of their choice. More specifically, the mentor will want to test whether the student has the skills and work ethics to complete a large coding project in a limited amount of time. The student will want to make sure that the mentor provides (useful) assistance when necessary. Therefore, students should select a mentor and a Summer of Code project before starting their work on a qualification task. The qualification task is often (but not necessarily) related to the selected project. For example, if your project will be to reverse engineer a new video codec, the qualification task may be to write a partial bitstream parser for that codec. If your project is to write a muxer for a container format, the qualification task may be to write the code to write the stream header.

There will be a second qualification task for every student: Pick a file of moderate size and reformat it in proper K&R style. The goal of this task is twofold: First it familiarizes students with the style that they will have to write their code in, second it demonstrates that students are able to submit patches from Git and go through our review process.

Applying

While you're working on your qualification task, apply at http://www.google-melange.com/. The degree (and detail) with which you've finished your qualification task will determine how likely your project is to be selected. In the past few years, students that completely finished their qualification task always got selected as Summer of Code students, but that may vary depending on the number of slots we are allotted by Google, and the number of students that apply.

1st Tier Project Proposals

1st tier project proposals are project ideas that are reasonably well defined AND have a mentor.

Hardware Acceleration API

Libav has two different ways to provide hardware acceleration wrapping.

One, deprecated, is to make a full decoder and signal its capabilities somehow, the other, called hwaccel, provides some API hooks to integrate the hardware acceleration in the stock decoder.

Currently hwaccel has some provision for transparent fallback, but lacks a clean way to setup and pass configuration options to it.

The project should improve hwaccel in this regard and add support for more accelerators.

  • Draft the API (that will require knowledge of libavcodec).
  • Extend the current implemented support to leverage the new features.
  • Implement Freescale VPU support.
  • Implement TI dce support.

Mentor: Luca Barbato

VP9 Native Decoder

The WebM team is refining the evolution of their VP8 codec using a number of additional tools and tunings. The project involves providing a baseline decoder as first step and optimizing it to be at least as fast as the libvpx one on one of the most used architectures (ARM and x86_64).

current draft

Mentor: Luca Barbato

WebP Lossless Native codec and WebP Lossy decoder

WebP is a promising image format that could supercede JPEG and PNG as common format for web images, it leverages VP8 for lossy encoding and uses a relatively simple original format for the lossless part.

The project is split in a number of easy tasks, such as writing a decoder for both the lossy and the lossless mode and a much harder task that is providing a good encoder for the lossless part, at least. (Exceptional praise for producing a good lossy encoder but it is not required.)

Mentor: Luca Barbato

Native DVD support

Most of the open source support for DVDs is available through libdvdread and libdvdnav. Currently there is some effort to unify them in a single library (libdvd5). An additional step would be factoring the non-interactive part of into the libav codebase to leverage even more code and leave as stand alone library the parts that require some kind of interaction.

Mentor: Diego Pettenò

Restructuring the MPEG video family of codecs

Many encoders and decoders currently use the MpegEncContext structure and its associated API. This structure is a huge monolithic blob which combines the features of all those encoders and decoders, thus making it very fragile and hard to understand.

The goal of this project is splitting out parts of the MpegEncContext API into self-contained structures that would be easier to grasp, while still allowing code reuse where appropriate.

Mentor: Anton Khirnov

Spin off build system into a separate project

Our build system is neat enough to make into a more general solution to be reused by other projects.

The goal of this project is to achieve exactly that. Intermediate steps will be reading, understanding and documenting the current build system, refactoring parts that can be generalized further and finally making a prototype implementation for libpostproc.

You will require skills in POSIX shell, GNU Make and a firm command of English.

Mentor: Diego Biurrun

Co-Mentor: Luca Barbato

The Grand Refactoring (working title)

Many parts of the libav codebase are still unnecessarily monolithic. This makes custom builds that only support a subset of the multitude of formats we have larger than they need to be and slows down the compiling time on multicore systems.

The goal of this project will be to locate parts that can be separated and refactor the code so that each subpart can be compiled standalone so as to not increase the size of a custom configuration without the part.

A suitable qualification task for this project is picking a simple encoder/decoder pair or some part of dsputil (harder) and splitting it cleanly.

Mentor: Diego Biurrun

Co-Mentor: Luca Barbato

Rewrite the ASF muxer and demuxer

Our current ASF muxer and demuxer were written using reverse engineering when the specification was not yet available. Because of that, they are hard to understand, contain many bugs and do not support the format fully. The goal of this project would be to rewrite those old parts of the ASF muxer and demuxer so that the new code:

  • conforms to the specification;
  • is clear and readable;
  • has less bugs;
  • supports all useful features of the format.

A qualification task would be adding support for chapters (called 'markers' in ASF) to the current ASF muxer.

Mentor: Anton Khirnov

Support for concatenation in avconv

One of the most important features still missing from our multimedia transcoder avconv is proper support for concatenating (joining) media streams. The goals of this project would be to:

  • add support for concatenating input streams when transcoding
  • add support for concatenating outputs of the filtergraphs when transcoding
  • (if time allows) add support for concatenating input streams when doing streamcopy.

A qualification task would be adding support for looping of arbitrary input files, to familiarize the student with avconv.

Mentor: Anton Khirnov

Rewrite the RealMedia demuxer

The current RM support is workable but it started from early attempts to support the yet to be documented format. Some features (like multirate files) are not supported properly either. The project aims to be a full blown rewrite to make a feature complete demuxer.

Mentor: Kostya Shishkov

Extend the libavfilter filter collection

Libavfilter, our library for audio and video filtering, is slowly reaching the level of mature and usable code, but the number of filters in it is still very small. The goal of this project would be to find the most useful filters (under appropriate licences) in projects like Avisynth, SoX, ImageMagick, etc. and port them to libavfilter. Depending on the student's experience level, he or she can also write the filters from scratch.

A qualification task would be writing a very simple filter, to make the student familiar with libavfilter.

Mentor: Anton Khirnov

Co-Mentor: Justin Ruggles

Adobe DNG Decoder (Basic Support)

Adobe Digital Negative (DNG) is an attempt at a universal file format for raw camera images. Most camera manufacturers have their own proprietary raw image format. Adobe provides tools for converting these to DNG with minimal or no loss of information for more reliable long-term support in a format with an open specification.

The project goal would be to add features required for basic support of DNG files. Some of these include:

  • test/improve TIFF and LJPEG 16-bit decoding support
  • implement both variants of JPEG-in-TIFF in the TIFF decoder
  • add basic handling for Bayer CFA pixel format(s), including demosaicing
  • conversion from camera colorspace to RGB
  • export of DNG/TIFF/Exif metadata

Resources:

Mentor: Justin Ruggles

Streaming protocol improvements

  • Implement support for producing data for Adobe HTTP Dynamic Streaming and MPEG DASH
  • Implement support for receiving the same

Alternatively you can also implement support for another new streaming protocol we currently don't support, or improve the ones we currently support.

Mentor: Martin Storsjö

Co-Mentor: Luca Barbato

2nd Tier Project Proposals

None of the following project have a mentor confirmed yet.

Language bindings

Provide bindings for non-C language. The languages can be any among Perl, Python, Ruby, Go and such.

Tasks

  • low-level abstraction with a 1:1 mapping to the C api.
  • high-level abstraction matching the wrapping language idoms.
  • example code leveraging it.

The bindings should be using only the public api.

Mentor: ???

Assembly Unit Testing Framework

  • Libav has a lot of assembly and not enough tests for it. Your job is to write a unit testing framework for assembly.
  • The framework should work across all supported architectures and operating systems.
  • The framework should measure exactly how fast an individual function is (e.g. using START/STOP_TIMER).
  • The framework should be able to test functions in isolation.
  • x264's checkasm can be used as a reference.
  • The qualification task will be to implement at least one unit test and have an idea of how to do the rest.

Mentor: Daniel Kang (Jumpyshoes on #libav-devel@chat.freenode.net; daniel.d.kang@gmail.com -- ping me on IRC and email me).

DTS-HD decoder

ETSI released the specification. Your job is to complete the existing decoder with the following features.

(1) Add support for mixed Core + DTS-HD stream structure
    (DtsCoreFrame+DtsHdFrame+DtsCoreFrame+DtsHdFrame+...), used by Blu-Ray main
    and commentary tracks.
(2) Add support for XXCh extension (6.1 and 7.1 channels).
(3) Add support for X96 extension (96khz).
(4) Add support for XLL extension (lossless).
(5) Add support for a pure DTS-HD stream structure
    (DtsHdFrame+DtsHdFrame+DtsHdFrame+...), used by Blu-Ray PiP tracks.
(6) Add support for XBR extension (extra bitrate).

Mentor: Benjamin Larsson

MPEG-4 ALS Roundup

This task is to update and enhance the existing ALS decoder as well as integrate and enhance the rudimentary encoder found at: https://github.com/justinruggles/FFmpeg-alsenc

Possible features are:

  • implement rls-lms in the decoder
  • do correct channel layout/sort handling in the decoder
  • update to current master
  • use codec private options
  • implement encode2(), setting pts and duration
  • document options and examples in encoders.texi
  • come up with a good set of encoding tests for FATE
  • implement mcc/channel sort in the encoder
  • implement rls-lms in the encoder
  • implement float support

Mentor: Justin Ruggles


Opus Decoder

Implement an independent Opus decoder using the publicly-available specification

  • The reference source code should only be used as a normative document reference when required (i.e. this should not just be a port of libopus)
  • Fully support Ogg/Opus mapping: https://wiki.xiph.org/OggOpus
  • Handle CELT, SILK, and Hybrid modes (including transitions)
  • Handle more than 2 channels
  • (optional) Make sure opus-in-mkv and opus-in-nut work

The initial SILK code had been written as result of the previous attempt to support Opus

Mentor: Justin Ruggles

On2 VP7 decoder

VP7 is a DCT-based video codec. At the moment, it seems to lack a reference implementation, but we do have a spec and may take hint from libavcodec's VP8 and VP6 decoder and libvpx. MPlayer can decode VP7 by loading a binary. To begin working on the project, one has to setup a reference decoder against which to compare our output. This can be done by either writing a wrapper for the binary or (maybe) hacking libvpx itself.

You might want to discuss with us how and where to start. Drop by on IRC if you need help. It is not as difficult as it sounds.

Mentor: Mashiat Sarker Shakkhar

Rewrite avserver

Libav needs a better system to serve streams, the current codebase had a number of design defect showing its age. The new avserver should be written from scratch, leveraging the knowledge piled up.

The implementation will be incrementally complex and possibly modular.

  • Write a simple rtsp, http, rtmp redirector (listen for publish/announce and rebroadcast the received streams)
  • Add the capability to serve on-demand content reading from a single path

Ideally the first implementation can be made using a poll/event loop and then moved to use threads.

avconv gained the ability to listen for incoming rtmp and rtsp connection as result of the previous year project.

Mentor: Luca Barbato

DTS-LBR decoder

This is the DTS equivalent to E-AC3 but not technically related to DTS/DTS-HD. It is found in the following specification. The task is to find a way of making the official decoder decode just the LBR stream (and not mix it into the main audio) and use that to verify decoder compliance of the decoder you wrote. The spec may be incomplete or require parts to be reverse engineered from the binary.

Mentor: ???