https://wiki.multimedia.cx/api.php?action=feedcontributions&user=Michael&feedformat=atomMultimediaWiki - User contributions [en]2024-03-28T10:03:43ZUser contributionsMediaWiki 1.39.5https://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=15012FFmpeg Summer of Code 20142014-02-08T21:29:52Z<p>Michael: /* Opus decoder */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== VDPAU filter ==<br />
<br />
'''Description:''' VDPAU is not only about hardware-accelerated decoding but also allows some postprocessing, most notably deinterlacing. Other features of the API include scaling, noise removal and a sharpening filter.<br />
<br />
'''Expected results:''' Implement a filter that allows using the postprocessing features as defined by the VDPAU API. This should particularly include the deinterlacer, ideally all features should be usable to allow comparing the quality and performance of different hardware and hardware vs. software.<br />
<br />
'''Prerequisites:''' C coding skills, you will need hardware that allows VDPAU post-processing.<br />
<br />
'''Qualification Task:''' Show that you are familiar with the FFmpeg filter system, either by porting one or more of the remaining postprocessing filters from the MPlayer compatibility layer (files ''vf_*pp*.c'' in ''libavfilter/libmpcodecs'', see for comparison commit [http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a2c547ff a2c547ff]) to a native filter or by implementing a very simple inverse telecine filter under LGPL that allows to reverse the effect of the telecine filter.<br />
<br />
'''Mentor:''' [[User:ce|Carl Eugen Hoyos]] (''cehoyos'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Basic servers for network protocols ==<br />
<br />
'''Description:''' libavformat contains client for various network protocols used in multimedia streaming: HTTP, RTMP, MMS, RTSP. Your work will be to implement the server side for one or several of these protocols.<br />
<br />
The libavformat framework is not designed to build general-purpose server applications with several clients, and nothing similar to the configuration features of real servers like Apache is expected, but libavformat should be able to stream a single predefined bytestream to/from a single client.<br />
<br />
Note: server support is already implemented for the receiving side of RTSP.<br />
<br />
'''Expected results:''' basic servers for network protocols capable of interoperating with third-party clients.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, network programming.<br />
<br />
'''Qualification Task:''' proof-of-concept server for one of the protocol, capable of interacting with a particular client in controlled circumstances; or anything network-related, e.g. fixing a Trac ticket.<br />
<br />
'''Mentor:''' [[User:Cigaes|Nicolas George]] (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Reynaldo|Reynaldo Verdejo]] (''reynaldo'' in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to TrueHD, one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
'''Expected results:''' a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding with a competetive compression rate<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' write a encoder that produces a valid bitstream that can be decoded without errors to silence.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported only through the external libopus library<br />
<br />
'''Expected results:''' a native opus decoder in ffmpeg<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, some knowledge about opus would be benefical<br />
<br />
'''Qualification Task:''' write part of the decoder, like write the bitstream decoding or header decoding and show that it is working.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
This task is difficult because it involves several design decisions that affect third party projects (video players).<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=15010FFmpeg Summer of Code 20142014-02-08T21:16:30Z<p>Michael: /* Opus decoder */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== VDPAU filter ==<br />
<br />
'''Description:''' VDPAU is not only about hardware-accelerated decoding but also allows some postprocessing, most notably deinterlacing. Other features of the API include scaling, noise removal and a sharpening filter.<br />
<br />
'''Expected results:''' Implement a filter that allows using the postprocessing features as defined by the VDPAU API. This should particularly include the deinterlacer, ideally all features should be usable to allow comparing the quality and performance of different hardware and hardware vs. software.<br />
<br />
'''Prerequisites:''' C coding skills, you will need hardware that allows VDPAU post-processing.<br />
<br />
'''Qualification Task:''' Show that you are familiar with the FFmpeg filter system, either by porting one or more of the remaining postprocessing filters from the MPlayer compatibility layer (files ''vf_*pp*.c'' in ''libavfilter/libmpcodecs'', see for comparison commit [http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a2c547ff a2c547ff]) to a native filter or by implementing a very simple inverse telecine filter under LGPL that allows to reverse the effect of the telecine filter.<br />
<br />
'''Mentor:''' [[User:ce|Carl Eugen Hoyos]] (''cehoyos'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Basic servers for network protocols ==<br />
<br />
'''Description:''' libavformat contains client for various network protocols used in multimedia streaming: HTTP, RTMP, MMS, RTSP. Your work will be to implement the server side for one or several of these protocols.<br />
<br />
The libavformat framework is not designed to build general-purpose server applications with several clients, and nothing similar to the configuration features of real servers like Apache is expected, but libavformat should be able to stream a single predefined bytestream to/from a single client.<br />
<br />
Note: server support is already implemented for the receiving side of RTSP.<br />
<br />
'''Expected results:''' basic servers for network protocols capable of interoperating with third-party clients.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, network programming.<br />
<br />
'''Qualification Task:''' proof-of-concept server for one of the protocol, capable of interacting with a particular client in controlled circumstances; or anything network-related, e.g. fixing a Trac ticket.<br />
<br />
'''Mentor:''' [[User:Cigaes|Nicolas George]] (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Reynaldo|Reynaldo Verdejo]] (''reynaldo'' in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to TrueHD, one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
'''Expected results:''' a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding with a competetive compression rate<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' write a encoder that produces a valid bitstream that can be decoded without errors to silence.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported only through the external libopus library<br />
<br />
'''Expected results:''' a native opus decoder in ffmpeg<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about opus.<br />
<br />
'''Qualification Task:''' write part of the decoder, like write the bitstream decoding or header decoding and show that it is working.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
This task is difficult because it involves several design decisions that affect third party projects (video players).<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=15009FFmpeg Summer of Code 20142014-02-08T21:14:12Z<p>Michael: /* TrueHD encoder */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== VDPAU filter ==<br />
<br />
'''Description:''' VDPAU is not only about hardware-accelerated decoding but also allows some postprocessing, most notably deinterlacing. Other features of the API include scaling, noise removal and a sharpening filter.<br />
<br />
'''Expected results:''' Implement a filter that allows using the postprocessing features as defined by the VDPAU API. This should particularly include the deinterlacer, ideally all features should be usable to allow comparing the quality and performance of different hardware and hardware vs. software.<br />
<br />
'''Prerequisites:''' C coding skills, you will need hardware that allows VDPAU post-processing.<br />
<br />
'''Qualification Task:''' Show that you are familiar with the FFmpeg filter system, either by porting one or more of the remaining postprocessing filters from the MPlayer compatibility layer (files ''vf_*pp*.c'' in ''libavfilter/libmpcodecs'', see for comparison commit [http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a2c547ff a2c547ff]) to a native filter or by implementing a very simple inverse telecine filter under LGPL that allows to reverse the effect of the telecine filter.<br />
<br />
'''Mentor:''' [[User:ce|Carl Eugen Hoyos]] (''cehoyos'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Basic servers for network protocols ==<br />
<br />
'''Description:''' libavformat contains client for various network protocols used in multimedia streaming: HTTP, RTMP, MMS, RTSP. Your work will be to implement the server side for one or several of these protocols.<br />
<br />
The libavformat framework is not designed to build general-purpose server applications with several clients, and nothing similar to the configuration features of real servers like Apache is expected, but libavformat should be able to stream a single predefined bytestream to/from a single client.<br />
<br />
Note: server support is already implemented for the receiving side of RTSP.<br />
<br />
'''Expected results:''' basic servers for network protocols capable of interoperating with third-party clients.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, network programming.<br />
<br />
'''Qualification Task:''' proof-of-concept server for one of the protocol, capable of interacting with a particular client in controlled circumstances; or anything network-related, e.g. fixing a Trac ticket.<br />
<br />
'''Mentor:''' [[User:Cigaes|Nicolas George]] (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Reynaldo|Reynaldo Verdejo]] (''reynaldo'' in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to TrueHD, one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
'''Expected results:''' a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding with a competetive compression rate<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' write a encoder that produces a valid bitstream that can be decoded without errors to silence.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
This task is difficult because it involves several design decisions that affect third party projects (video players).<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=15008FFmpeg Summer of Code 20142014-02-08T21:13:10Z<p>Michael: /* TrueHD encoder */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== VDPAU filter ==<br />
<br />
'''Description:''' VDPAU is not only about hardware-accelerated decoding but also allows some postprocessing, most notably deinterlacing. Other features of the API include scaling, noise removal and a sharpening filter.<br />
<br />
'''Expected results:''' Implement a filter that allows using the postprocessing features as defined by the VDPAU API. This should particularly include the deinterlacer, ideally all features should be usable to allow comparing the quality and performance of different hardware and hardware vs. software.<br />
<br />
'''Prerequisites:''' C coding skills, you will need hardware that allows VDPAU post-processing.<br />
<br />
'''Qualification Task:''' Show that you are familiar with the FFmpeg filter system, either by porting one or more of the remaining postprocessing filters from the MPlayer compatibility layer (files ''vf_*pp*.c'' in ''libavfilter/libmpcodecs'', see for comparison commit [http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a2c547ff a2c547ff]) to a native filter or by implementing a very simple inverse telecine filter under LGPL that allows to reverse the effect of the telecine filter.<br />
<br />
'''Mentor:''' [[User:ce|Carl Eugen Hoyos]] (''cehoyos'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Basic servers for network protocols ==<br />
<br />
'''Description:''' libavformat contains client for various network protocols used in multimedia streaming: HTTP, RTMP, MMS, RTSP. Your work will be to implement the server side for one or several of these protocols.<br />
<br />
The libavformat framework is not designed to build general-purpose server applications with several clients, and nothing similar to the configuration features of real servers like Apache is expected, but libavformat should be able to stream a single predefined bytestream to/from a single client.<br />
<br />
Note: server support is already implemented for the receiving side of RTSP.<br />
<br />
'''Expected results:''' basic servers for network protocols capable of interoperating with third-party clients.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, network programming.<br />
<br />
'''Qualification Task:''' proof-of-concept server for one of the protocol, capable of interacting with a particular client in controlled circumstances; or anything network-related, e.g. fixing a Trac ticket.<br />
<br />
'''Mentor:''' [[User:Cigaes|Nicolas George]] (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Reynaldo|Reynaldo Verdejo]] (''reynaldo'' in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to TRueHD, one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
'''Expected results:''' a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding with a competetive compression rate<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' write a encoder that produces a valid bitstream that can be decoded without errors to silence.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
This task is difficult because it involves several design decisions that affect third party projects (video players).<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=15005FFmpeg Summer of Code 20142014-02-08T16:23:01Z<p>Michael: /* Mentored Projects */ backup mentor for Symmetric-key block ciphers</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== VDPAU filter ==<br />
<br />
'''Description:''' VDPAU is not only about hardware-accelerated decoding but also allows some postprocessing, most notably deinterlacing. Other features of the API include scaling, noise removal and a sharpening filter.<br />
<br />
'''Expected results:''' Implement a filter that allows using the postprocessing features as defined by the VDPAU API. This should particularly include the deinterlacer, ideally all features should be usable to allow comparing the quality and performance of different hardware and hardware vs. software.<br />
<br />
'''Prerequisites:''' C coding skills, you will need hardware that allows VDPAU post-processing.<br />
<br />
'''Qualification Task:''' Show that you are familiar with the FFmpeg filter system, either by porting one or more of the remaining postprocessing filters from the MPlayer compatibility layer (files ''vf_*pp*.c'' in ''libavfilter/libmpcodecs'', see for comparison commit [http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a2c547ff a2c547ff]) to a native filter or by implementing a very simple inverse telecine filter under LGPL that allows to reverse the effect of the telecine filter.<br />
<br />
'''Mentor:''' [[User:ce|Carl Eugen Hoyos]] (''cehoyos'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Basic servers for network protocols ==<br />
<br />
'''Description:''' libavformat contains client for various network protocols used in multimedia streaming: HTTP, RTMP, MMS, RTSP. Your work will be to implement the server side for one or several of these protocols.<br />
<br />
The libavformat framework is not designed to build general-purpose server applications with several clients, and nothing similar to the configuration features of real servers like Apache is expected, but libavformat should be able to stream a single predefined bytestream to/from a single client.<br />
<br />
Note: server support is already implemented for the receiving side of RTSP.<br />
<br />
'''Expected results:''' basic servers for network protocols capable of interoperating with third-party clients.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, network programming.<br />
<br />
'''Qualification Task:''' proof-of-concept server for one of the protocol, capable of interacting with a particular client in controlled circumstances; or anything network-related, e.g. fixing a Trac ticket.<br />
<br />
'''Mentor:''' [[User:Cigaes|Nicolas George]] (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
This task is difficult because it involves several design decisions that affect third party projects (video players).<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14999FFmpeg Summer of Code 20142014-02-08T04:39:34Z<p>Michael: /* Hardware Acceleration API Software/Tracing Implementation */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14998FFmpeg Summer of Code 20142014-02-08T04:25:14Z<p>Michael: /* Misc Libavfilter extension */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14997FFmpeg Summer of Code 20142014-02-08T04:24:40Z<p>Michael: /* MXF Demuxer Improvements */ remove TBA & possibly</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14996FFmpeg Summer of Code 20142014-02-08T04:01:30Z<p>Michael: move H.264 Multiview Video Coding (MVC) to unmentored projects</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:'''<br />
<br />
'''Backup mentor:''' <br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14995FFmpeg Summer of Code 20142014-02-08T03:58:12Z<p>Michael: move "hardware Accelerated Video Encoding with VA-API" to unmentored projects</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14994FFmpeg Summer of Code 20142014-02-08T03:53:45Z<p>Michael: /* JPEG 2000 */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14993FFmpeg Summer of Code 20142014-02-08T03:53:07Z<p>Michael: /* VC-1 interlaced */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.<br />
<br />
'''Expected results:'''<br />
* Implement missing parts of interlacing<br />
* Make more reference samples bit-exact<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14992FFmpeg Summer of Code 20142014-02-08T03:47:40Z<p>Michael: /* JPEG 2000 */ lots of improvments</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
<br />
'''Expected results:''' A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.<br />
<br />
'''Qualification Task:''' Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, good knowledge of Jpeg-2000<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14991FFmpeg Summer of Code 20142014-02-08T03:40:04Z<p>Michael: /* glplay */ remove as we alreadx have a opengl device</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintainance. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]] and [[FFmpeg Summer Of Code 2010|2010]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admin:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
* '''FFmpeg GSoC Backup Admin:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
[[File:Mmspg-epfl-ch-double-camera.jpg|right|link=]]<br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
[[File:Animated PNG example bouncing beach ball.png|right|link=]]<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
[[File:Lavfi-gsoc-filter-vintage-illustration.jpg|right|link=]]<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
[[File:PostProc.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
[[File:350px-Bayer_pattern_on_sensor.svg.png|right|link=]]<br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 Audio Lossless Coding (ALS) encoder ==<br />
<br />
[[File:Showwaves_green.png|240px|right|link=]]<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
'''Backip Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
[[File:Hardware.jpg|right|link=]]<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvements ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See [https://trac.ffmpeg.org/ticket/2776 ticket #2776] in our bug tracker, and [https://trac.ffmpeg.org/ticket/2776 ticket #1916] contains additional relevant information.<br />
<br />
Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket [https://trac.ffmpeg.org/ticket/2776 ticket #2776] should demux correctly. Add a test case in [http://ffmpeg.org/fate.html FATE]. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the [http://www.smpte-ra.org/mdd/RP224v10-publication-20081215.xls official RP224 Excel document]. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
[[File:Opus.png|right|link=]]<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
[[File:Jpeg2000.jpg|left|link=]]<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14976FFmpeg Summer of Code 20142014-02-06T16:29:00Z<p>Michael: </p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
<br clear="all"/><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' <br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
<br clear="all"/><br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
<br clear="all"/><br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14975FFmpeg Summer of Code 20142014-02-06T16:27:24Z<p>Michael: /* DCA Encoder improvments */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:'''<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' <br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14974FFmpeg Summer of Code 20142014-02-06T16:27:09Z<p>Michael: /* DTS / DCA Decoder Improvements */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:'''<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' <br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14973FFmpeg Summer of Code 20142014-02-06T16:26:29Z<p>Michael: /* MXF Demuxer Improvements */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:'''<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' <br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14972FFmpeg Summer of Code 20142014-02-06T16:26:01Z<p>Michael: /* FFv1 P frame support */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:'''<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' <br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14971FFmpeg Summer of Code 20142014-02-06T16:25:34Z<p>Michael: /* Hardware Accelerated Video Encoding with VA-API */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' <br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14970FFmpeg Summer of Code 20142014-02-06T16:22:26Z<p>Michael: add Symmetric-key block ciphers</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
== Symmetric-key block ciphers ==<br />
<br />
'''Description:'''<br />
FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.<br />
<br />
'''Specification:'''<br />
* http://tools.ietf.org/search/rfc2144<br />
* https://www.schneier.com/twofish.html<br />
* http://tools.ietf.org/html/rfc3713<br />
<br />
'''Expected Results:'''<br />
* Implement CAST-128, Twofish and Camellia in C<br />
* Support for ECB and CBC modes<br />
* Ensure code works with all key sizes supported by an algorithm<br />
* Ensure code is conformant to the standard and passes test vectors<br />
* Ensure code does not attempt any invalid read or write using valgrind<br />
* Simple C-level optimization to get comparable speed with other crypto libraries<br />
<br />
'''Prerequisites:'''<br />
C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.<br />
<br />
'''Qualification Task:''' Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.<br />
<br />
'''Mentor:''' Giorgio Vazzana (''Holden'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:'''<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14969FFmpeg Summer of Code 20142014-02-06T14:18:18Z<p>Michael: /* Hardware Acceleration (hwaccel) API v2 */ drop due to lack of mentor</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14968FFmpeg Summer of Code 20142014-02-06T14:16:26Z<p>Michael: /* DTS / DCA Decoder Improvements */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement. <br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14967FFmpeg Summer of Code 20142014-02-06T13:27:09Z<p>Michael: /* DCA Encoder improvments */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14966FFmpeg Summer of Code 20142014-02-06T13:26:52Z<p>Michael: /* DTS / DCA Improvements */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Decoder Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan/merbzt'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14965FFmpeg Summer of Code 20142014-02-06T13:25:20Z<p>Michael: DCA encoder</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder, but its missing several features<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== DCA Encoder improvments ==<br />
<br />
'''Description:''' Add more complete multichannel support, subband adpcm support and optimize the decorrelation transform. A [http://wiki.multimedia.cx/index.php?title=Mirror specification] is available.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.<br />
<br />
'''Qualification Task:''' Add 3.0 / 3.1 support and fix the channel order for 5.1<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14963FFmpeg Summer of Code 20142014-02-06T04:47:29Z<p>Michael: /* H.264 Multiview Video Coding (MVC) */ imrpove wording, link to existing work in progress</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' <br />
MVC is used in 3D Blu-ray disks, but FFmpeg is missing a decoder which supports it. The goal of this project is to add support for MVC and 3D-Blu rays.<br />
Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list]. There also exists a [http://article.gmane.org/gmane.comp.video.ffmpeg.devel/174155 patch] and a [https://github.com/kodabb/libav/commits/MVC_orig_clean git branch] which are in rather bad shape but could be used as basis for this project.<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Understanding of H.264<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14962FFmpeg Summer of Code 20142014-02-06T04:12:04Z<p>Michael: /* Qualification Tasks */ imho better wording</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14961FFmpeg Summer of Code 20142014-02-06T04:08:22Z<p>Michael: /* Getting Started */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14960FFmpeg Summer of Code 20142014-02-06T03:45:12Z<p>Michael: /* Introduction */ make timeline a link</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC Timeline] for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
# '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
# '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
# '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
# '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=User:Michael&diff=14959User:Michael2014-02-06T03:41:12Z<p>Michael: dont obfuscate my email, its written all over the web anyway</p>
<hr />
<div>Michael Niedermayer, developer and project maintainer for FFmpeg.<br />
He's ''michaelni'' on the #ffmpeg-devel freenode IRC channel.<br />
you can send him mail at michaelni@gmx.at</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14955FFmpeg Summer of Code 20142014-02-05T19:05:21Z<p>Michael: move hwaccel deode v2 task to unmentored projects</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the GSoC Timeline for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration API Software/Tracing Implementation ==<br />
<br />
'''Description:''' Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps and in part since it requires specific operating systems and hardware.<br />
<br />
The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and the data they get to allow for easy comparison and thus regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what apitrace https://github.com/apitrace/apitrace does for OpenGL.<br />
<br />
A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git<br />
<br />
'''Qualification Task:''' Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable<br />
<br />
'''Mentor:''' Reimar Döffinger (''reimar'' in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)<br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=User:Michael&diff=14953User:Michael2014-02-05T19:00:36Z<p>Michael: add email</p>
<hr />
<div>Michael Niedermayer, developer and project maintainer for FFmpeg.<br />
He's ''michaelni'' on the #ffmpeg-devel freenode IRC channel.<br />
you can send him mail at michaelni gmx at</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14951FFmpeg Summer of Code 20142014-02-05T18:28:39Z<p>Michael: drop outdated qual task /* Hardware Accelerated Video Encoding with VA-API */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the GSoC Timeline for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14950FFmpeg Summer of Code 20142014-02-05T18:28:12Z<p>Michael: drop outdated qual task /* Hardware Acceleration (hwaccel) API v2 */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the GSoC Timeline for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14949FFmpeg Summer of Code 20142014-02-05T18:20:08Z<p>Michael: ben agreed to mentor again if a sufficiently qualified student is found /* DTS / DCA Improvements */</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the GSoC Timeline for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2014&diff=14945FFmpeg Summer of Code 20142014-02-05T00:05:59Z<p>Michael: FFmpeg google summer of code 2014 page</p>
<hr />
<div>This page is on wiki.multimedia.cx due to trac.ffmpeg.org being down for maintaince. It might be moved back later.<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year.<br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2014 Google Summer of Code 2014]. See the GSoC Timeline for important dates.<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2014/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. Browse the [https://trac.ffmpeg.org FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' TBA<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly Kieran Kunhya (''kierank'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs, the goal of this project is to change that and add support. The little bouncing ball animation shown to the right is such a APNG file.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf)<br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe and image demuxer.<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== FFv1 P frame support ==<br />
<br />
'''Description:''' FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.<br />
<br />
'''Expected results:''' State of the art P frame support in the FFv1 encoder and decoder implementation.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.<br />
<br />
'''Qualification Task:''' Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' TBA<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.<br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check [https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Nicolas George (''Cigaes'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor:''' TBA, possibly [[User:Suxen_drol|Peter Ross]] (''pross-au'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
'''Description:'''<br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:'''<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwaccel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' TBA, possibly [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly Tushar Gohad<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvements ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://trac.ffmpeg.org/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Mentor:''' TBA, possibly Benjamin Larsson (''merbanan'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA, possibly [[User:Stefanosa|Stefano Sabatini]] (''saste'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
== MXF Demuxer Improvements ==<br />
<br />
'''Description:''' FFmpeg's MXF demuxer needs a proper, compact way to map EssenceContainer? ULs to WrappingKind?. See ticket #2776. I wrote stuff in ticket #1916 which is also relevant.<br />
<br />
The gist of this is that essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (like 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.<br />
<br />
'''Expected results:''' The sample in ticket #2776 demuxes fine and there's a test case in FATE for it. The solution should grow libavformat by no more than 32 KiB.<br />
<br />
'''Prerequisites:''' C coding skills, basic familiarity with git.<br />
<br />
'''Qualification Task:''' Investigate if there may be a compact way of representing the UL -> WrappingKind? mapping specified in the official RP224 Excel document. The tables takes up about half a megabyte verbatim, which is unacceptable in a library as large as libavformat.<br />
<br />
'''Mentor:''' TBA, possibly Tomas Härdin (''thardin'' in #ffmpeg-devel on Freenode IRC)<br />
<br />
'''Backup Mentor:''' TBA<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your Own Project Idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://trac.ffmpeg.org/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14547FFmpeg Summer of Code 20132013-04-02T11:44:54Z<p>Michael: /* DTS / DCA */ Better title</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
'''Backup mentor:''' Kieran Kunhya (kierank on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
'''Backup Mentor:''' Hendrik Leppkes (nevcairiel on IRC) <br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, knowledge about transform based audio coding would be usefull.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA Improvments ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14546FFmpeg Summer of Code 20132013-04-02T11:44:09Z<p>Michael: Ben volunteered to mentor DTS / DCA</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
'''Backup mentor:''' Kieran Kunhya (kierank on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
'''Backup Mentor:''' Hendrik Leppkes (nevcairiel on IRC) <br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvements ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, knowledge about transform based audio coding would be usefull.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Mentor:''' Benjamin Larsson (''merbanan'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14543FFmpeg Summer of Code 20132013-04-01T23:07:58Z<p>Michael: /* AAC */ better title</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
'''Backup mentor:''' Kieran Kunhya (kierank on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== AAC Improvments ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, knowledge about transform based audio coding would be usefull.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14542FFmpeg Summer of Code 20132013-04-01T23:06:32Z<p>Michael: /* AAC */ remove duplicate text</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
'''Backup mentor:''' Kieran Kunhya (kierank on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, knowledge about transform based audio coding would be usefull.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14541FFmpeg Summer of Code 20132013-04-01T23:05:45Z<p>Michael: /* AAC */ qual task formating</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
'''Backup mentor:''' Kieran Kunhya (kierank on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Qualification Task:''' See the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head or fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, knowledge about transform based audio coding would be usefull.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14540FFmpeg Summer of Code 20132013-04-01T22:54:24Z<p>Michael: Move AAC up to mentored projects, baptiste agreed to mentor it</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
'''Backup mentor:''' Kieran Kunhya (kierank on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and handing them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec; this incurs extra work in the client application, which tends to be duplicated over several client applications; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, knowledge about transform based audio coding would be usefull.<br />
<br />
'''Mentor:''' Baptiste Coudurier (''bcoudurier'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14537FFmpeg Summer of Code 20132013-03-29T18:11:38Z<p>Michael: /* H.264 Multiview Video Coding (MVC) */ add backup mentor</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
'''Backup mentor:''' Kieran Kunhya (kierank on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and hanging them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec ; this incurs extra work in the client application, which tends to be duplicated over several client applications ; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle ; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream ; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14536FFmpeg Summer of Code 20132013-03-29T17:02:49Z<p>Michael: /* Bayer RGB colorspaces */ Prerequisites</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Prerequisites''': C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and hanging them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec ; this incurs extra work in the client application, which tends to be duplicated over several client applications ; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle ; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream ; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14535FFmpeg Summer of Code 20132013-03-29T13:00:06Z<p>Michael: /* Bayer RGB colorspaces */ clarify that we want high quality transforms in the gsoc project</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement high quality bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and hanging them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec ; this incurs extra work in the client application, which tends to be duplicated over several client applications ; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle ; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream ; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14534FFmpeg Summer of Code 20132013-03-29T12:58:55Z<p>Michael: /* Bayer RGB colorspaces */ replace TBD by some specific qualification task</p>
<hr />
<div>[[Image:Ffmpeg-logo-gsoc.jpg]]<br />
<br />
= Introduction =<br />
<br />
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.<br />
<br />
[https://developers.google.com/open-source/soc/ Google Summer of Code (GSoC)] is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated to several past editions ([[FFmpeg Summer Of Code 2006|2006]], [[FFmpeg Summer Of Code 2007|2007]], [[FFmpeg Summer Of Code 2008|2008]], [[FFmpeg Summer Of Code 2009|2009]], [[FFmpeg Summer Of Code 2010|2010]], and [[FFmpeg / Libav Summer Of Code 2011|2011]]), and we are looking forward to being involved this year. <br />
<br />
This is our ideas page for [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013].<br />
<br />
== Information for Students ==<br />
<br />
=== Getting Started ===<br />
<br />
0. '''Get to know FFmpeg.''' If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel] mailing-list, visiting our IRC channels (''#ffmpeg-devel'' and ''#ffmpeg''), and exploring the codebase and the development workflow. Feel free to [[#Contacting_FFmpeg|contact us]] if you have any questions.<br />
<br />
1. '''Find a project.''' Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentors have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level.<br />
<br />
2. '''Contact us.''' If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the appointed mentors so that the task can be claimed.<br />
<br />
3. '''Apply.''' Student proposal period begins April 22, 2013 at 19:00 UTC and ends May 3rd at 19:00 UTC. See the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#2._What_is_the_program_timeline GSoC timeline] for additional information.<br />
<br />
=== Qualification Tasks ===<br />
<br />
In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to finish a project.<br />
<br />
The qualification task is usually shown in the project description. Contact the appointed mentors for assistance on getting a related qualification task or if you want to propose your own. See a list of [[Small FFmpeg Tasks]] or browse the [https://ffmpeg.org/trac/ffmpeg FFmpeg Bug Tracker] for qualification task ideas.<br />
<br />
=== Contacting FFmpeg ===<br />
<br />
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:<br />
<br />
* '''Mailing-list:''' [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel]<br />
* '''IRC:''' ''#ffmpeg-devel'' on Freenode<br />
* '''FFmpeg GSoC Admins:''' [[User:Stefanosa|Stefano Sabatini]] and [[User:Llogan|Lou Logan]]<br />
<br />
You can also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.<br />
<br />
= Mentored Projects =<br />
<br />
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the appointed mentor(s) to get more information about the project and the requested qualification task.<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
<div class="floatright">[[Image:Mmspg-epfl-ch-double-camera.jpg]]</div><br />
<br />
'''Description:''' MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
'''Backup mentor:''' [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write or port one or more filters<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]] (''saste'' in IRC)<br />
<br />
'''Backup mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Update the API to get rid of the clumsy internal text representation of styles<br />
* Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.<br />
* BONUS: if everything goes well, the student will be allowed to add basic support for teletext<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification Task:''' write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.<br />
<br />
'''Mentor:''' [[User:Ubitux|Clément Bœsch]] (''ubitux'' on IRC)<br />
<br />
'''Backup Mentor:''' Nicolas George (''Cigaes'' on IRC)<br />
<br />
== Postproc optimizations ==<br />
<br />
<div class="floatright">[[Image:PostProc.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libpostproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Bayer RGB colorspaces ==<br />
<br />
<div class="floatright">[[Image:350px-Bayer_pattern_on_sensor.svg.png ]]</div><br />
<br />
'''Description:''' Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.<br />
<br />
'''Expected Results:'''<br />
* Rebase existing patches<br />
* Implement bayer transformations in libswscale (plain C)<br />
* Add bayer formats to the libavutil pixfmt enumeration routines<br />
* SIMD optimizations of the libswscale transformations<br />
* Complete PhotoCINE demuxer to support Bayer format; (or another format of your choosing)<br />
<br />
Optional goodies:<br />
* Extend TIFF decoder to support DNG-Bayer format<br />
* Support a popular proprietary camera format (many to choose from; see dcraw project)<br />
<br />
'''Qualification Task''': Implement a simple and working Bayer->RGB transform in libswscale<br />
<br />
'''Mentor''': [[User:Suxen_drol|Peter Ross]] (''pross-au'' on IRC)<br />
<br />
'''Backup Mentor''': [[User:Michael|Michael Niedermayer]] (''michaelni'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification Task:''' Add floating point support to MPEG-4 ALS decoder<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]] (''durandal_1707'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Acceleration (hwaccel) API v2 ==<br />
<br />
<div class="floatright">[[Image:Hardware.jpg]]</div><br />
<br />
'''Description:''' FFmpeg supports hardware accelerated decoding through the internal hwacel API. Currently supported system hardware acceleration APIs are VA-API (Linux), DXVA2 (Windows) and VDA (MacOS X). However, the current approach requires client applications to allocate the underlying resources (e.g. hardware surfaces and context) themselves, and hanging them over to FFmpeg. This incurs a few limitations: this is not scalable to new codecs, i.e. this requires new tokens for each newly supported codec ; this incurs extra work in the client application, which tends to be duplicated over several client applications ; and this prevents efficient fallback to software decoding mode if the hardware cannot handle a particular codec specification.<br />
<br />
The goal of this project is to revamp the FFmpeg Hardware Acceleration API so that hardware resources are allocated and managed in the library, thus requiring the client application to only provide a single hardware context/device handle ; provide a way to fallback early to software decoding mode if the underlying hardware won't be able to handle the bitstream ; and make it possible to select a hardware accelerator by ID and not polluting the PixelFormats namespace.<br />
<br />
'''Expected results:'''<br />
* FFmpeg core library (libavcodec):<br />
** Core API extensions and improvements<br />
*** Add open/close hooks in a way that is backwards compatible with hwaccel v1 enabled applications<br />
*** Add new tokens describing hardware accelerators<br />
*** Add new flags exposing HW capabilities like download/upload<br />
*** Investigate the benefits or impacts to provide a global map/unmap capability to FFmpeg video buffers<br />
** Port hwaccels to v2 infrastructure<br />
*** Port VA-API decoders to v2 infrastructure<br />
*** Validate that VA-API decoders still work with existing applications supporting hwaccel v1<br />
*** Provide download capability through ''vaGetImage()''<br />
*** Validate that ffplay can support this feature with minor changes, and definitely no change to the existing SDL renderer<br />
*** Port VDPAU decoders to hwaccel v2 (optional), and investigate ways to preserve compatibility with older applications<br />
<br />
* FFmpeg applications:<br />
** Integrate hardware acceleration into ffplay<br />
*** Create a video-output (VO) infrastructure to ffplay<br />
*** Port the SDL renderer to the new VO infrastructure<br />
*** Add support for VA-API: VA renderer through ''vaPutSurface()'', add -hwaccel option to select "vaapi" renderer<br />
*** Add support for VDPAU (optional): VDPAU renderer through ''VdpPresentationQueueDisplay()''<br />
** Integrate hardware acceleration into ffmpeg<br />
*** Add support for VA-API: use the VA/DRM API for headless (no-X display) decoding, use libudev to determine the device to use<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"><br />
<br />
== Hardware Accelerated Video Encoding with VA-API ==<br />
<br />
'''Description:''' FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new ''hwaccel'' hooks are needed and two operational modes are possible: either ''(i)'' driver or hardware pack headers themselves, or ''(ii)'' lattitude is left to perform this task at the FFmpeg library level.<br />
<br />
'''Expected results:''' Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.<br />
* MPEG-2 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** (Optionally) add support for interlaced contents<br />
* H.264 encoding:<br />
** Add basic encoding with I/P frames (handle the ''-g'' option)<br />
** Add support for B frames (handle the ''-bf'' option)<br />
** Add support for constant bitrate (CBR, i.e. ''maxrate == bitrate'' and ''bufsize'' set)<br />
** Add support for constant QP (CQP, i.e. handle the ''-cqp'' option)<br />
** Add support for more than one reference frame, while providing/using API to query the hardware capabilities<br />
** Work on HRD conformance. May require to write an independent tool to assess that<br />
** (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.<br />
* FFmpeg applications:<br />
** Define common hwaccel interface for encoding<br />
** Add initial support for hardware accelerated encoding to the ''ffmpeg'' application<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems, hardware supporting VA-API for encoding.<br />
<br />
'''Qualification Task:''' Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.<br />
<br />
'''Mentor:''' [[User:Gwenole_Beauchesne|Gwenole Beauchesne]] (''__gb__'' on IRC)<br />
<br />
<br clear="all"/><br />
<br />
= Unmentored Projects =<br />
<br />
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also [[#Your_Own_Idea|propose their own project]].<br />
<br />
== glplay ==<br />
<br />
<div class="floatleft">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket [https://ffmpeg.org/trac/ffmpeg/ticket/1920 #1920]<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many [http://samples.mplayerhq.hu/V-codecs/VP7/ VP7 samples] are in the wild, but no open-source decoder exists although a [http://multimedia.cx/mirror/VP7_Data_Format_and_Decoder_Overview.pdf specification] exists. Write a decoder that reuses as much as possible of existing FFmpeg code: it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== VP8L ==<br />
<br />
'''Description:''' [[VP8L]] is a lossless format used in WebP. There is no support for this in FFmpeg.<br />
<br />
== Your own idea ==<br />
<br />
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our [https://ffmpeg.org/trac/ffmpeg/ bug tracker]. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.<br />
<br />
Students can discuss an idea in the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing-list], the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins [[User:Stefanosa|Stefano Sabatini]] or [[User:Llogan|Lou Logan]] for more information.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14434FFmpeg Summer of Code 20132013-03-15T23:05:45Z<p>Michael: Postproc</p>
<hr />
<div>Google Summer of Code is a program that offers students stipends to write code for open source projects. FFmpeg participated to several past editions, and this page is about the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Google Summer of Code 2013 edition].<br />
In particular, see [http://www.google-melange.com/gsoc/events/google/gsoc2013 timeline].<br />
<br />
Application period starts on March 18, and March 29 at 19:00 UTC is the application deadline for interested organizations.<br />
Please finish up this page before then.<br />
<br />
= Proposing Ideas =<br />
<br />
If you have a project idea please contact a developer first at the #ffmpeg-devel IRC channel on Freenode or via the [http://ffmpeg.org/contact.html ffmpeg-devel mailing list]. A good source of ideas is the [https://ffmpeg.org/trac/ffmpeg/ FFmpeg bug tracker] and [[FFmpeg_Summer_of_Code_2012|FFmpeg Summer of Code 2012 Ideas List]].<br />
<br />
When adding an idea follow this template for consistency:<br />
<br />
<pre><br />
== Task Title ==<br />
<br />
'''Description:''' A few sentences or a short paragraph describing the task.<br />
<br />
'''Expected results:''' Bulleted list or paragraph describing what the student is expected to achieve.<br />
<br />
'''Prerequisites:''' Skills or knowledge required by student.<br />
<br />
'''Qualification Task:''' Small task student should perform to demonstrate understanding.<br />
<br />
'''Mentor:''' List mentor and backup mentor if there is one and contact info such as IRC name or email address.<br />
</pre><br />
<br />
= Mentored tasks =<br />
<br />
== H.264 Multiview Video Coding (MVC) ==<br />
<br />
'''Description:''' MVC samples exist and the codec is used on Bluray media, but FFmpeg is missing a decoder. Since this task also consists of some changes in the current architecture, it is especially important that this task is discussed on the [http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ffmpeg-devel mailing list].<br />
<br />
'''Expected results:''' Create MVC decoder and add a test for the FFmpeg Automated Testing Environment (FATE).<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' Michael Niedermayer<br />
<br />
== Animated Portable Network Graphics (APNG) ==<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor:''' [[User:Pbm|Paul B Mahol]]<br />
<br />
== Misc Libavfilter extension ==<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification task:''' write or port one or more filter<br />
<br />
'''Mentor:''' [[User:Stefanosa|Stefano Sabatini]]. Backup mentor: Clément Bœsch (''ubitux'' on IRC).<br />
<br />
<br clear="all"><br />
<div class="floatright">[[Image:Three-two pulldown diagram (telecine).png]]</div><br />
<br />
== Inverse telecine filter ==<br />
<br />
'''Description:''' Libavfilter has all kind of filters but is really lacking an IVTC one. It's a must-have we have been delaying for too long.<br />
<br />
Various very good filters already exists in other projects, notably TIVTC (basically the reference filter), which have been ported/rewritten recently in a portable version (VIVTC). Some other solutions exists in the MPlayer project (something like 4 or 5 filters) but their efficiency is questionable. The VLC project also has its own complete toolkit for inverse telecine which might be interesting to look at, especially for its very complete documentation on the topic.<br />
<br />
'''Expected results:''' Write/port a complete and efficient inverse telecine filter to libavfilter.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some familiarity with the DVD encoding process and the image processing techniques would be a bonus but are not strictly required.<br />
<br />
'''Qualification task:''' Make a comparative report about the current filters in various projects (MPlayer, TIVTC, VIVTC, VLC, Handbrake, ...) with all kind of different inputs, notably mixed and badly broken content. This is a very important step that has to be done before writing any code, and will be extremely useful as a future reference.<br />
<br />
'''Mentor:''' Clément Bœsch (''ubitux'' on IRC). + Derek?<br />
<br />
<br clear="all"><br />
<br />
== Subtitles ==<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carry an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Write the support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Improve the API to facilitates a proper integration of subtitles into libavfilter.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification task:''' write one subtitles demuxer or muxer, and do some improvements to the different layers of the subtitles (to make sure the whole chain is understood).<br />
<br />
'''Mentor:''' Clément Bœsch (''ubitux'' on IRC).<br />
<br />
== Postproc optimizations ==<br />
<br />
'''Description:''' FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H263 amongth others). The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).<br />
<br />
'''Expected results:'''<br />
* Convert all gcc inline asm in libportproc to YASM.<br />
* Restructure the code so that it works with block sizes compatible with modern SIMD.<br />
* Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.<br />
<br />
'''Prerequisites:''' C coding skills, good x86 assembly coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification task:''' convert 1 or 2 MMX2 functions to SSE2 and AVX2.<br />
<br />
'''Mentor:''' Michael Niedermayer (''michaelni'' on IRC).<br />
<br />
== glplay ==<br />
<br />
<div class="floatright">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
= Unmentored tasks =<br />
<br />
== AAC ==<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
== DTS / DCA ==<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder and an experimental DTS encoder. Both are missing some features.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket #1920<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
* DTS encoder improvements<br />
<br />
== MPEG-4 ALS encoder ==<br />
<br />
<div class="floatright">http://upload.wikimedia.org/wikipedia/commons/e/e9/ATunes.png</div><br />
<div class="floatright">[http://commons.wikimedia.org/wiki/File%3AATunes.png]</div><br />
<br />
'''Description:''' <br />
A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on [https://github.com/justinruggles/FFmpeg-alsenc.git github]. For this task, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the [http://www.nue.tu-berlin.de/menue/forschung/projekte/beendete_projekte/mpeg-4_audio_lossless_coding_als/parameter/en/#230252 reference codec and specifications]. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.<br />
<br />
'''Expected results:''' <br />
<br />
Mandatory objectives<br />
<br />
* Update the existing encoder to fit into the current codebase.<br />
* Ensure conformance of the encoder by verifying using the reference codec and generate a test case for FATE.<br />
* Push the encoder into the codebase after iterating through the review process on the developing mailing list.<br />
<br />
Optional goodies<br />
<br />
* Ensure the FFmpeg decoder processes all generated files without warnings.<br />
* Enhance the rudimentary feature set of the encoder.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.<br />
<br />
'''Qualification task:''' Some good idea needed!<br />
<br />
'''Mentor:''' Mentor needed!<br />
<br />
<br clear="all"><br />
<br />
== TrueHD encoder ==<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
== Opus decoder ==<br />
<br />
<div class="floatright">[[Image:Opus.png]]</div><br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== HEVC / H.265 ==<br />
<br />
'''Description:''' The specification was finished, FFmpeg needs a decoder for this new format.<br />
* Write a basic decoder supporting I, P, and, only if time permits, B slices.<br />
* It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.<br />
As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.<br />
<br />
== VC-1 interlaced ==<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
== JPEG 2000 ==<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
== VP7 ==<br />
<br />
'''Description:''' Not many VP7 samples are in the wild, but no open-source decoder exists although a specification exists. Write a decoder that reuses as much as possible of existing FFmpeg code, it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14407FFmpeg Summer of Code 20132013-03-06T21:59:30Z<p>Michael: add myself as mentor /* H264 MVC */</p>
<hr />
<div>FFmpeg Summer of Code Ideas List<br />
<br />
March 29 at 19:00 UTC is the application deadline for interested organizations<br />
Please finish up this page before then.<br />
<br />
= Proposing Ideas =<br />
<br />
If you have a project idea please contact a developer first at the #ffmpeg-devel IRC channel on Freenode or via the [http://ffmpeg.org/contact.html ffmpeg-devel mailing list]. A good source of ideas is the [https://ffmpeg.org/trac/ffmpeg/ FFmpeg bug tracker] and [[FFmpeg_Summer_of_Code_2012|FFmpeg Summer of Code 2012 Ideas List]].<br />
<br />
When adding an idea follow this template for consistency:<br />
<br />
== Example Title ==<br />
<br />
'''Description:''' A few sentences or a short paragraph describing the task.<br />
<br />
'''Expected results:''' Bulleted list or paragraph describing what the student is expected to achieve.<br />
<br />
'''Prerequisites:''' Skills or knowledge required by student.<br />
<br />
'''Mentor:''' List mentor and backup mentor if there is one and contact info such as IRC name or email address.<br />
<br />
= GSoC task proposal ideas =<br />
<br />
== Audio codecs ==<br />
<br />
<center>[[Image:Audio_Spectrum.png]]</center><br />
<br />
=== AAC ===<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
=== DTS / DCA ===<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder and an experimental DTS encoder. Both are missing some features.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket #1920<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
* DTS encoder improvements<br />
<br />
=== MPEG-4 ALS encoder ===<br />
<br />
'''Description:''' An ALS decoder was implemented several years ago, an encoder is still missing.<br />
<br />
=== TrueHD encoder ===<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
=== Opus decoder ===<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== Video codecs ==<br />
<br />
<center>[[Image:BBB-Scenes.jpg]]</center><br />
<br />
=== HEVC / H265 ===<br />
<br />
'''Description:''' The specification was finished, FFmpeg needs a decoder for this new format.<br />
* Write a basic decoder supporting I, P, and, only if time permits, B slices.<br />
* It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.<br />
As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.<br />
<br />
=== H264 MVC ===<br />
<br />
'''Description:''' MVC samples exist, the codec is used on Bluray media, FFmpeg is missing a decoder. Since this task also consists of some changes in the current architecture, it is especially important that this task is discussed on the ffmpeg-devel mailing list.<br />
As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
'''Mentor:''' Michael Niedermayer<br />
<br />
=== VC-1 interlaced ===<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
=== Animated Portable Network Graphics (APNG) ===<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
<div class="floatright">[[Image:Animated PNG example bouncing beach ball.png]]</div><br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
'''Qualification Task:''' Implement format autodetection for imagepipe & image demuxer<br />
<br />
'''Mentor: [[User:Pbm|Paul B Mahol]]'''<br />
<br />
=== JPEG 2000 ===<br />
<br />
<div class="floatleft">[[Image:Jpeg2000.jpg]]</div><br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
<br clear="all"><br />
<br />
=== VP7 ===<br />
<br />
'''Description:''' Not many VP7 samples are in the wild, but no open-source decoder exists although a specification exists. Write a decoder that reuses as much as possible of existing FFmpeg code, it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== Other ==<br />
<br />
=== glplay ===<br />
<br />
<div class="floatright">[[Image:Opengl_logo.jpg]]</div><br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
'''Mentor:''' TBD Backup: Reimar Döffinger<br />
<br />
<br clear="all"><br />
<br />
=== Misc Libavfilter extension ===<br />
<br />
<div class="floatright">[[Image:Lavfi-gsoc-filter-vintage-illustration.jpg]]</div><br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification task:''' write or port one or more filter<br />
<br />
'''Mentor:''' Stefano Sabatini (''saste'' on IRC). Backup: Clément Bœsch (''ubitux'' on IRC).<br />
<br />
<br clear="all"><br />
<div class="floatright">[[Image:Three-two pulldown diagram (telecine).png]]</div><br />
<br />
=== Inverse telecine filter ===<br />
<br />
'''Description:''' Libavfilter has all kind of filters but is really lacking an IVTC one. It's a must-have we have been delaying for too long.<br />
<br />
Various very good filters already exists in other projects, notably TIVTC (basically the reference filter), which have been ported/rewritten recently in a portable version (VIVTC). Some other solutions exists in the MPlayer project (something like 4 or 5 filters) but their efficiency is questionable. The VLC project also has its own complete toolkit for inverse telecine which might be interesting to look at, especially for its very complete documentation on the topic.<br />
<br />
'''Expected results:''' Write/port a complete and efficient inverse telecine filter to libavfilter.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some familiarity with the DVD encoding process and the image processing techniques would be a bonus but are not strictly required.<br />
<br />
'''Qualification task:''' Make a comparative report about the current filters in various projects (MPlayer, TIVTC, VIVTC, VLC, Handbrake, ...) with all kind of different inputs, notably mixed and badly broken content. This is a very important step that has to be done before writing any code, and will be extremely useful as a future reference.<br />
<br />
'''Mentor:''' Clément Bœsch (''ubitux'' on IRC). + Derek?<br />
<br />
<br clear="all"><br />
<br />
=== Subtitles ===<br />
<br />
'''Description:''' FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carry an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.<br />
<br />
<center>[[Image:Subtitles-sensei.jpg]]</center><br />
<br />
'''Expected results:'''<br />
* Write the support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray.<br />
* Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.<br />
* Improve the API to facilitates a proper integration of subtitles into libavfilter.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.<br />
<br />
'''Qualification task:''' write one subtitles demuxer or muxer, and do some improvements to the different layers of the subtitles (to make sure the whole chain is understood).<br />
<br />
'''Mentor:''' Clément Bœsch (''ubitux'' on IRC).<br />
<br />
[[Category:FFmpeg]]</div>Michaelhttps://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_of_Code_2013&diff=14365FFmpeg Summer of Code 20132013-02-23T15:42:42Z<p>Michael: /* JPEG 2000 */</p>
<hr />
<div>FFmpeg Summer of Code Ideas List<br />
<br />
= Proposing Ideas =<br />
<br />
If you have a project idea please contact a developer first at the #ffmpeg-devel IRC channel on Freenode or via the [http://ffmpeg.org/contact.html ffmpeg-devel mailing list]. A good source of ideas is the [https://ffmpeg.org/trac/ffmpeg/ FFmpeg bug tracker] and [[FFmpeg_Summer_of_Code_2012|FFmpeg Summer of Code 2012 Ideas List]].<br />
<br />
When adding an idea follow this template for consistency:<br />
<br />
== Example Title ==<br />
<br />
'''Description:''' A few sentences or a short paragraph describing the task.<br />
<br />
'''Expected results:''' Bulleted list or paragraph describing what the student is expected to achieve.<br />
<br />
'''Prerequisites:''' Skills or knowledge required by student.<br />
<br />
'''Mentor:''' List mentor and backup mentor if there is one and contact info such as IRC name or email address.<br />
<br />
= GSoC task proposal ideas =<br />
<br />
== Audio codecs ==<br />
<br />
=== AAC ===<br />
<br />
'''Description:''' FFmpeg contains an AAC encoder and decoder, both of them can be improved in various ways. This is enough work for more than one GSoC project, so one part of your submission would be to define on which task exactly you want to work.<br />
* AAC LD decoder<br />
* AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples<br />
* AAC SSR decoder<br />
* AAC 960/120 MDCT window<br />
* AAC multi-channel encoding<br />
See also the FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head fixing one or more existing bugs are possible qualification tasks.<br />
<br />
=== DTS / DCA ===<br />
<br />
'''Description:''' FFmpeg contains a DTS decoder and an experimental DTS encoder. Both are missing some features.<br />
* DTS-HD decoder improvements: A possible qualification task is to implement ticket #1920<br />
** Add support for X96 extension (96khz)<br />
** Add support for XLL extension (lossless)<br />
** Add support for pure DTS-HD streams that do not contain a DTS core<br />
** Add support for multiple assets<br />
** Add support for LBR extension<br />
* DTS encoder improvements<br />
<br />
=== MPEG-4 ALS encoder ===<br />
<br />
'''Description:''' An ALS decoder was implemented several years ago, an encoder is still missing.<br />
<br />
=== TrueHD encoder ===<br />
<br />
'''Description:''' FFmpeg currently does not support encoding to one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.<br />
<br />
=== Opus decoder ===<br />
<br />
'''Description:''' Opus decoding is currently supported through the external libopus library<br />
* Write a native decoder, continue working on the existing unfinished implementation<br />
A possible qualification task is to port the existing incomplete decoder to current git head and improve it to show that you are capable of working on this task.<br />
<br />
== Video codecs ==<br />
<br />
=== GotoMeeting Decoder ===<br />
<br />
'''Description:''' The [[GoToMeeting_Codec]] has been figured out and we need a decoder for it. We don't have samples for G2M1 so this decoder will focus on G2M2, G2M3 and G2M4 versions.<br />
<br />
=== HEVC / H265 ===<br />
<br />
'''Description:''' The specification was finished, FFmpeg needs a decoder for this new format.<br />
* Write a basic decoder supporting I, P, and, only if time permits, B slices.<br />
* It does not need to be ASM/SIMD optimized but its high level structure must permit such optimizations to be easily added later.<br />
As a qualification task you need to implement parsing headers and maybe a bit beyond that to demonstrate that you are qualified and understand the HEVC specification. This project requires a solid understanding of video coding and C, it's not something for the average SOC student.<br />
<br />
=== H264 MVC ===<br />
<br />
'''Description:''' MVC samples exist, the codec is used on Bluray media, FFmpeg is missing a decoder. Since this task also consists of some changes in the current architecture, it is especially important that this task is discussed on the ffmpeg-devel mailing list.<br />
As qualification you have to do some work that demonstrates your understanding of MVC and that is a subpart of the whole MVC implementation.<br />
<br />
=== VC-1 interlaced ===<br />
<br />
'''Description:''' The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts.<br />
* Implement missing interlace features<br />
* Make more reference samples bit-exact<br />
As a qualification task, you should try to find a bug in the current decoder implementation and fix it.<br />
<br />
=== Animated Portable Network Graphics (APNG) ===<br />
<br />
'''Description:''' FFmpeg currently does not support Animated PNGs.<br />
<br />
'''Specification:''' https://wiki.mozilla.org/APNG_Specification<br />
<br />
'''Expected results:'''<br />
* APNG demuxer<br />
** implement robust probing:<br />
*** PNG images are not misdetected as APNG animations<br />
*** APNG animations are not misdetected as PNG images<br />
** splits stream into sensible packets (so they can be easily reused in APNG muxer)<br />
** survives fuzzing (zzuf)<br />
** add FATE coverage, coverage should be at least 70%<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG decoder<br />
** use existing PNG decoder code (write decoder in same file)<br />
** implement parsing of all APNG chunks (acTL, fcTL, fdAT)<br />
** error handling<br />
** survives fuzzing (zzuf) <br />
** add test for FATE, coverage should be at least 75%<br />
** CRC checksum validation<br />
** test code under valgrind so no invalid reads/writes happen<br />
<br />
* APNG muxer && APNG encoder<br />
** use existing PNG encoder code (write encoder in same file)<br />
** write compliant files, make sure they play correctly in major web browsers that support APNG<br />
** add test for FATE<br />
<br />
'''Mentor: [[User:Pbm|Paul B Mahol]]'''<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems.<br />
<br />
=== JPEG 2000 ===<br />
<br />
'''Description:''' FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples.<br />
Work on an issue (for example from the bug tracker) as a qualification task to show that you are capable of improving the codec implementation.<br />
<br />
=== VP7 ===<br />
<br />
'''Description:''' Not many VP7 samples are in the wild, but no open-source decoder exists although a specification exists. Write a decoder that reuses as much as possible of existing FFmpeg code, it is likely that functions of the existing decoders for On2-based formats will be useful.<br />
<br />
== Other ==<br />
<br />
=== glplay ===<br />
<br />
'''Description:''' The SDL library that is used by FFplay has some deficiencies, adding OpenGL output to FFplay should allow for better performance (and less bugs at least for some hardware / driver combinations). This could be a new application (glplay), but it is probably simpler to extend ffplay to use OpenGL. You can use code from MPlayer's OpenGL vo module which may be relicensed under the LGPL.<br />
<br />
=== Misc Libavfilter extension ===<br />
<br />
'''Description:''' Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch. <br />
<br />
Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.<br />
<br />
Some ideas for more filters:<br />
* a frequency filtering domain filter relying on the FFT utils in libavcodec<br />
* a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI<br />
* a lua scripting filter, which allows to implement filtering custom logic in lua<br />
<br />
For more ideas check:<br />
[https://ffmpeg.org/trac/ffmpeg/query?status=new&status=open&status=reopened&component=avfilter&col=id&col=summary&col=status&col=type&col=priority&col=component&col=version&order=priority trac libavfilter tickets].<br />
<br />
'''Expected results:''' Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.<br />
<br />
'''Prerequisites:''' C coding skills, familiarity with git/source code control systems. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.<br />
<br />
'''Qualification task:''' write or port one or more filter<br />
<br />
'''Mentor:''' Stefano Sabatini (''saste'' on IRC).<br />
<br />
[[Category:FFmpeg]]</div>Michael