FFmpeg Summer Of Code 2006: Difference between revisions
m (→VC-1) |
Gamester17 (talk | contribs) No edit summary |
||
(15 intermediate revisions by 6 users not shown) | |||
Line 1: | Line 1: | ||
[http://www.google.com/ Google] | [http://www.google.com/ Google] sponsored SoC 2006 (Summer of Code 2006), which was the second [http://code.google.com/summerofcode.html Summer of Code] sponsored by Google. [http://code.google.com/soc/ffmpeg/about.html FFmpeg] has been selected as one of the approved mentoring organizations. If you have any questions, feel free to contact the FFmpeg mentoring administrator, Mike Melanson, at (mike -at- multimedia.cx). This page contains a few project ideas (which, by no means, limits any prospective proposals). | ||
'''Note! For lists of ongoing and new projects please see:''' | |||
*[[FFmpeg Wishlist]] for more tasks or ideas. | |||
**[[FFmpeg Summer Of Code 2007|FFmpeg's Google Summer of Code 2007 list of tasks]] for more suggestions/requests (ideas for developers). | |||
*[http://bugzilla.mplayerhq.hu/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=FFmpeg&content= FFmpeg bugs] for bugs in FFmpegs (codecs) that you can help fix or add addition information/samples to. | |||
*[[:Category:Formats missing in FFmpeg]] for formats not implemented in ffmpeg yet | |||
== Applying == | == Applying == | ||
'''The application process for the 2006 SoC is closed and the projects have been chosen.''' | |||
If you are interested in participating in the Google Summer of Code you will need to register at the official SoC site linked above and file an official application. Traditionally, top applications are fairly detailed. Look at this application for inspiration [http://vfer.sourceforge.net/cgi-bin/index.cgi?page=writeup]. The [http://www.drupal.org/ Drupal] project has also prepared guidelines: | If you are interested in participating in the Google Summer of Code you will need to register at the official SoC site linked above and file an official application. Traditionally, top applications are fairly detailed. Look at this application for inspiration [http://vfer.sourceforge.net/cgi-bin/index.cgi?page=writeup]. The [http://www.drupal.org/ Drupal] project has also prepared guidelines: | ||
* http://drupal.org/node/59037 | * http://drupal.org/node/59037 | ||
Line 8: | Line 18: | ||
== VC-1 == | == VC-1 == | ||
'''Accepted''' | |||
[[VC-1]], a.k.a. Microsoft Windows Media Video 3 or 9, is widely used and should see even wider use due to the introduction of [[HD-DVD]]. The format for this codec is open and known, but so far unimplemented in FFmpeg. This task entails implementing the VC-1 codec based on the emerging [[Understanding VC-1|open specification Wiki document]] and forthcoming roadmap. This is a big task but will have very clearly defined milestones, e.g., the first major goal is to decode I-frames, and to get that point there will be a number of smaller goals, each with its own set of unit tests. | [[VC-1]], a.k.a. Microsoft Windows Media Video 3 or 9, is widely used and should see even wider use due to the introduction of [[HD-DVD]]. The format for this codec is open and known, but so far unimplemented in FFmpeg. This task entails implementing the VC-1 codec based on the emerging [[Understanding VC-1|open specification Wiki document]] and forthcoming roadmap. This is a big task but will have very clearly defined milestones, e.g., the first major goal is to decode I-frames, and to get that point there will be a number of smaller goals, each with its own set of unit tests. | ||
Line 26: | Line 38: | ||
== AAC == | == AAC == | ||
'''Accepted''' | |||
AAC is quickly becoming the new MP3. FFmpeg wants its own independent implementation. This task entails implementing a new, optimized AAC decoder in a similar manner as outlined in the [[FFmpeg Summer Of Code#VC-1|VC-1]] section, based on the emerging [[Understanding AAC|open specification Wiki document]]. | AAC is quickly becoming the new MP3. FFmpeg wants its own independent implementation. This task entails implementing a new, optimized AAC decoder in a similar manner as outlined in the [[FFmpeg Summer Of Code#VC-1|VC-1]] section, based on the emerging [[Understanding AAC|open specification Wiki document]]. | ||
''Mentor: Mike Melanson, | ''Mentor: Oded Shimon; Backup mentors: Benjamin Larsson, Mike Melanson'' | ||
== AC3 == | |||
'''Accepted''' | |||
Implement a LGPL/BSD/MIT licensed (E)AC3 decoder (Dolby Digital(Plus)) for FFmpeg based on the specifications found at http://wiki.multimedia.cx/index.php?title=A52. | |||
* The main objective is a fairly optimized floating point version of the decoder; expected time for that is one month. | |||
* The second objective is a fixed point version of the decoder; expected time for that is two months. | |||
Sample files for this task can be found at http://samples.mplayerhq.hu/A-codecs/AC3-samples/. If sample files for Enhanced AC-3 (Dolby Digital Plus) turn up before the task starts, the enhanced part of the AC-3 is to be included in the task. The resulting code should be clean and commented. It should use already existing infrastructure in FFmpeg when possible. | |||
''Mentor: Benjamin Larsson'' | |||
== Vorbis == | |||
'''Accepted''' | |||
Implement a [[Vorbis]] encoder | |||
''Mentor: Oded Shimon'' | |||
== AMR == | |||
'''Accepted''' | |||
Implement an LGPL [[AMR]] decoder (encoder). AMR (Adaptive Multi Rate) is the codec used in several new mobiles. Specifications can be found here [http://www.3gpp.org/ftp/Specs/html-info/26-series.htm]. A decoder for both AMR narrow band and wide band is the goal. The expected time for this task is 2 months. | |||
''Mentor: Benjamin Larsson'' | |||
== Profiling and Optimization == | |||
Profile and optimize memory usage | |||
== FFmpeg Refactoring And Test Suite == | == FFmpeg Refactoring And Test Suite == | ||
Multimedia programs tend to be highly modular in design and FFmpeg is no exception. However, it does not make the best use of independent modules. The major task in refactoring FFmpeg modules will be to reorganize code so that each individual codec or muxer/demuxer module can be easily enabled and disabled at compile time. This task also entails creating | Multimedia programs tend to be highly modular in design and FFmpeg is no exception. However, it does not make the best use of independent modules. The major task in refactoring FFmpeg modules will be to reorganize code so that each individual codec or muxer/demuxer module can be easily enabled and disabled at compile time. This task also entails creating a test suite that can automatically enable each module, one at a time, and validate that FFmpeg still builds and works. | ||
''Mentor: Mike Melanson'' | ''Mentor: Mike Melanson'' | ||
== Improving libavformat == | == Improving libavformat == | ||
libavformat is the interface of FFmpeg that is responsible for splitting apart encoded audio and video data from multimedia files (demuxing) and putting it together in new multimedia files (muxing). While libavcodec -- the FFmpeg component that encodes and decodes audio and video data -- enjoys [http://ffmpeg. | libavformat is the interface of FFmpeg that is responsible for splitting apart encoded audio and video data from multimedia files (demuxing) and putting it together in new multimedia files (muxing). While libavcodec -- the FFmpeg component that encodes and decodes audio and video data -- enjoys [http://ffmpeg.mplayerhq.hu/projects.html widespread use among an impressive array of multimedia projects] libavformat has not seen the same level of adoption. This task would entail investigating how to improve the libavformat API, how it interacts with client applications and input layers, developing proof of concept code for a new API and working to port existing muxers and demuxers to the new API. | ||
''Mentor: Mike Melanson'' | ''Mentor: Mike Melanson'' | ||
== Dirac == | == Dirac == | ||
[[Dirac]] is an experimental codec developed by [[BBC]] as an open standard | [[Dirac]] is an experimental codec developed by [[BBC]] as an open standard. It employs a wide range of advanced compression tecniques and shares some features with [[Snow]]. | ||
Implementing this codec based on the upcoming new specification from [http://dirac.sf.net Dirac site] will follow a roadmap/milestone approach similar to the one that will be used on the [[VC-1]] project. | Implementing this codec based on the upcoming new specification from [http://dirac.sf.net Dirac site] will follow a roadmap/milestone approach similar to the one that will be used on the [[VC-1]] project. | ||
Line 47: | Line 89: | ||
* Good C knowledge | * Good C knowledge | ||
* Basic knowledge of compression | * Basic knowledge of compression techniques in multimedia | ||
''Mentor: Luca Barbato'' | ''Mentor: Luca Barbato'' | ||
== NUT == | == NUT == | ||
There are already many | There are already many widely used and well specified container formats like [[Matroska]] or [[Ogg]] that are open and free for use. Each of them have some shortcomings due design decisions that make them non optimal for a number of applications. | ||
[[NUT]] is designed to be simple to implement and parse, easy to seek to precise position without unnecessary reads even on partial files, error resistant and with the lowest possible overhead. | [[NUT]] is designed to be simple to implement and parse, easy to seek to precise position without unnecessary reads, even on partial files, error resistant, and with the lowest possible overhead. | ||
This Summer of Code project consists | This Summer of Code project consists of updating the current ancient ffmpeg implementation to be in line with the | ||
[http://www1.mplayerhq.hu/cgi-bin/cvsweb.cgi/~checkout~main/DOCS/tech/mpcf.txt?rev=1.155&content-type=text/plain&cvsroot=main current specification], benchmark it against other implementations of [[NUT]] and other container formats. | [http://www1.mplayerhq.hu/cgi-bin/cvsweb.cgi/~checkout~main/DOCS/tech/mpcf.txt?rev=1.155&content-type=text/plain&cvsroot=main current specification], and benchmark it against other implementations of [[NUT]] and other container formats. | ||
Another part of the project will be helping the team | Another part of the project will be helping the team to improve the documentation available; this will be mostly a side effect of the implementation task. | ||
Roadmap: | Roadmap: | ||
* Study the current specification and clarify it, putting it in a more verbose and understandable form. | * Study the current specification and clarify it, putting it in a more verbose and understandable form. | ||
** Conversion to rst | ** Conversion to rst or docbook isn't really required, but would be greatly appreciated. | ||
* Update the demuxer using libnut produced files as testcases | * Update the demuxer using libnut-produced files as testcases | ||
** Testcase 1: it should demux | ** Testcase 1: it should demux the complete file correctly as sequential reads | ||
** Testcase 2: it should seek correctly the complete file | ** Testcase 2: it should seek correctly in the complete file | ||
** Testcase 3: same as 1 but with corrupted file | ** Testcase 3: same as 1 but with corrupted file | ||
** Testcase 4: same as 2 but with corrupted file | ** Testcase 4: same as 2 but with corrupted file | ||
* Update the muxer using libnut and ffnut demuxer to validate the produced files | * Update the muxer using libnut and ffnut demuxer to validate the produced files | ||
** Make sure that the interleaving rules are respected. | ** Make sure that the interleaving rules are respected. | ||
Requirements: | Requirements: | ||
Line 79: | Line 119: | ||
''Mentor: Luca Barbato'' | ''Mentor: Luca Barbato'' | ||
== | == Snow == | ||
[[ | [[Snow]] is a lossy and lossless video codec featuring wavelet transform and overlapping block based motion compensation. It shares with [[Dirac]] the choice of wavelets but it is quite simpler and with less optional features. At the present the ffmpeg implementation is partially optimized for x86, AMD64 and PPC and usable on high end systems. | ||
This project could be split in two macrotasks: | This project could be split in two macrotasks: | ||
* Improve the optimizations and refine the implementation in order to have | * Improve the optimizations and refine the implementation in order to have FFmpeg playing [[Snow]] in more constrained environments. Most of the work will be related to avoid cache trashing and vectorize the code. | ||
* Write a formal specification of the codec and document the implementation. | * Write a formal specification of the codec and document the implementation. | ||
Line 93: | Line 133: | ||
For this project you must know: | For this project you must know: | ||
* How to profile an application using oprofile, valgrind and the internal timers available in | * How to profile an application using oprofile, valgrind and the internal timers available in FFmpeg | ||
* SIMD theory and how to vectorize in practice | * SIMD theory and how to vectorize certain algorithms in practice | ||
* DocBook, rst, or doxygen for the documentation steps | * DocBook, rst, or doxygen for the documentation steps | ||
Line 101: | Line 141: | ||
* Basic knowledge of the math beside the transformations in use. | * Basic knowledge of the math beside the transformations in use. | ||
* | * AltiVec, MMX/SSE, VIS or any other vector extension assembly/C intrinsics for your favourite arch. | ||
** | ** AltiVec would preferably be written as intrinsics | ||
** SSE/MMX/3dNow! code as inline assembly. | ** SSE/MMX/3dNow! code as inline assembly. | ||
''Mentor: Luca Barbato'' | ''Mentor: Luca Barbato'' | ||
== MP3 == | == MP3 == | ||
Line 134: | Line 154: | ||
== Psychoacoustics == | == Psychoacoustics == | ||
Implement a good [[Psychoacoustic Model|psychoacoustic model]]. Support the usage of this model from the AC3, MP2 and other audio encoders. | Implement a good [[Psychoacoustic Model|psychoacoustic model]]. Support the usage of this model from the AC3, MP2 and other audio encoders. | ||
==See Also== | |||
*[[FFmpeg Wishlist]] for more tasks or ideas. | |||
**[[FFmpeg Summer Of Code 2007|FFmpeg's Google Summer of Code 2007 list of tasks]] for more suggestions/requests (ideas for developers). | |||
*[http://bugzilla.mplayerhq.hu/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=FFmpeg&content= FFmpeg bugs] for bugs in FFmpegs (codecs) that you can help fix or add addition information/samples to. | |||
*[[:Category:Formats missing in FFmpeg]] for formats not implemented in FFmpeg yet | |||
[[Category:FFmpeg]] |
Latest revision as of 04:30, 3 January 2008
Google sponsored SoC 2006 (Summer of Code 2006), which was the second Summer of Code sponsored by Google. FFmpeg has been selected as one of the approved mentoring organizations. If you have any questions, feel free to contact the FFmpeg mentoring administrator, Mike Melanson, at (mike -at- multimedia.cx). This page contains a few project ideas (which, by no means, limits any prospective proposals).
Note! For lists of ongoing and new projects please see:
- FFmpeg Wishlist for more tasks or ideas.
- FFmpeg's Google Summer of Code 2007 list of tasks for more suggestions/requests (ideas for developers).
- FFmpeg bugs for bugs in FFmpegs (codecs) that you can help fix or add addition information/samples to.
- Category:Formats missing in FFmpeg for formats not implemented in ffmpeg yet
Applying
The application process for the 2006 SoC is closed and the projects have been chosen.
If you are interested in participating in the Google Summer of Code you will need to register at the official SoC site linked above and file an official application. Traditionally, top applications are fairly detailed. Look at this application for inspiration [1]. The Drupal project has also prepared guidelines:
Further, there is a Google Group that includes much discussion of what it takes to be accepted into this program. Good luck!
VC-1
Accepted
VC-1, a.k.a. Microsoft Windows Media Video 3 or 9, is widely used and should see even wider use due to the introduction of HD-DVD. The format for this codec is open and known, but so far unimplemented in FFmpeg. This task entails implementing the VC-1 codec based on the emerging open specification Wiki document and forthcoming roadmap. This is a big task but will have very clearly defined milestones, e.g., the first major goal is to decode I-frames, and to get that point there will be a number of smaller goals, each with its own set of unit tests.
The decoder should be clean, simple, readable, and efficient. It should incorporate existing FFmpeg facilities where appropriate.
Major goals/milestones for this project include:
- create a skeleton decoder module to ensure that you understand how the module interacts with the FFmpeg system
- decode progressive I-frame
- decode progressive P-frame
- decode progressive B-frame
- decode interlaced I-frame
- decode interlaced P-frame
- decode interlaced B-frame
- SIMD optimizations
Note that the progressive frame decodes are the most important steps for decoding existing media available via the internet. Interlaced decoding is less important, but is still necessary. SIMD optimizations (optimizing certain critical functions using CPU instructions like MMX, SSE2 and Altivec) is a "nice to have" feature in the end but not strictly part of this endeavor.
Mentor: Mike Melanson
AAC
Accepted
AAC is quickly becoming the new MP3. FFmpeg wants its own independent implementation. This task entails implementing a new, optimized AAC decoder in a similar manner as outlined in the VC-1 section, based on the emerging open specification Wiki document.
Mentor: Oded Shimon; Backup mentors: Benjamin Larsson, Mike Melanson
AC3
Accepted
Implement a LGPL/BSD/MIT licensed (E)AC3 decoder (Dolby Digital(Plus)) for FFmpeg based on the specifications found at http://wiki.multimedia.cx/index.php?title=A52.
- The main objective is a fairly optimized floating point version of the decoder; expected time for that is one month.
- The second objective is a fixed point version of the decoder; expected time for that is two months.
Sample files for this task can be found at http://samples.mplayerhq.hu/A-codecs/AC3-samples/. If sample files for Enhanced AC-3 (Dolby Digital Plus) turn up before the task starts, the enhanced part of the AC-3 is to be included in the task. The resulting code should be clean and commented. It should use already existing infrastructure in FFmpeg when possible.
Mentor: Benjamin Larsson
Vorbis
Accepted
Implement a Vorbis encoder
Mentor: Oded Shimon
AMR
Accepted
Implement an LGPL AMR decoder (encoder). AMR (Adaptive Multi Rate) is the codec used in several new mobiles. Specifications can be found here [2]. A decoder for both AMR narrow band and wide band is the goal. The expected time for this task is 2 months.
Mentor: Benjamin Larsson
Profiling and Optimization
Profile and optimize memory usage
FFmpeg Refactoring And Test Suite
Multimedia programs tend to be highly modular in design and FFmpeg is no exception. However, it does not make the best use of independent modules. The major task in refactoring FFmpeg modules will be to reorganize code so that each individual codec or muxer/demuxer module can be easily enabled and disabled at compile time. This task also entails creating a test suite that can automatically enable each module, one at a time, and validate that FFmpeg still builds and works.
Mentor: Mike Melanson
Improving libavformat
libavformat is the interface of FFmpeg that is responsible for splitting apart encoded audio and video data from multimedia files (demuxing) and putting it together in new multimedia files (muxing). While libavcodec -- the FFmpeg component that encodes and decodes audio and video data -- enjoys widespread use among an impressive array of multimedia projects libavformat has not seen the same level of adoption. This task would entail investigating how to improve the libavformat API, how it interacts with client applications and input layers, developing proof of concept code for a new API and working to port existing muxers and demuxers to the new API.
Mentor: Mike Melanson
Dirac
Dirac is an experimental codec developed by BBC as an open standard. It employs a wide range of advanced compression tecniques and shares some features with Snow. Implementing this codec based on the upcoming new specification from Dirac site will follow a roadmap/milestone approach similar to the one that will be used on the VC-1 project.
Requirements:
- Good C knowledge
- Basic knowledge of compression techniques in multimedia
Mentor: Luca Barbato
NUT
There are already many widely used and well specified container formats like Matroska or Ogg that are open and free for use. Each of them have some shortcomings due design decisions that make them non optimal for a number of applications. NUT is designed to be simple to implement and parse, easy to seek to precise position without unnecessary reads, even on partial files, error resistant, and with the lowest possible overhead. This Summer of Code project consists of updating the current ancient ffmpeg implementation to be in line with the current specification, and benchmark it against other implementations of NUT and other container formats. Another part of the project will be helping the team to improve the documentation available; this will be mostly a side effect of the implementation task.
Roadmap:
- Study the current specification and clarify it, putting it in a more verbose and understandable form.
- Conversion to rst or docbook isn't really required, but would be greatly appreciated.
- Update the demuxer using libnut-produced files as testcases
- Testcase 1: it should demux the complete file correctly as sequential reads
- Testcase 2: it should seek correctly in the complete file
- Testcase 3: same as 1 but with corrupted file
- Testcase 4: same as 2 but with corrupted file
- Update the muxer using libnut and ffnut demuxer to validate the produced files
- Make sure that the interleaving rules are respected.
Requirements:
- Good C knowledge
- Technical writing skills
Mentor: Luca Barbato
Snow
Snow is a lossy and lossless video codec featuring wavelet transform and overlapping block based motion compensation. It shares with Dirac the choice of wavelets but it is quite simpler and with less optional features. At the present the ffmpeg implementation is partially optimized for x86, AMD64 and PPC and usable on high end systems.
This project could be split in two macrotasks:
- Improve the optimizations and refine the implementation in order to have FFmpeg playing Snow in more constrained environments. Most of the work will be related to avoid cache trashing and vectorize the code.
- Write a formal specification of the codec and document the implementation.
A complete roadmap will be decided together with the candidates.
For this project you must know:
- How to profile an application using oprofile, valgrind and the internal timers available in FFmpeg
- SIMD theory and how to vectorize certain algorithms in practice
- DocBook, rst, or doxygen for the documentation steps
- Basic knowledge of the math beside the transformations in use.
- AltiVec, MMX/SSE, VIS or any other vector extension assembly/C intrinsics for your favourite arch.
- AltiVec would preferably be written as intrinsics
- SSE/MMX/3dNow! code as inline assembly.
Mentor: Luca Barbato
MP3
Implement a simple MP3 encoder, the encoder should be a fixedpoint implementation and using the already existing infrastructure in ffmpeg when possible. A bitstream compatible encoder should take 1 month to complete and 2 more months for a fairly optimized good sounding encoder.
Mentor: Benjamin Larsson
Psychoacoustics
Implement a good psychoacoustic model. Support the usage of this model from the AC3, MP2 and other audio encoders.
See Also
- FFmpeg Wishlist for more tasks or ideas.
- FFmpeg's Google Summer of Code 2007 list of tasks for more suggestions/requests (ideas for developers).
- FFmpeg bugs for bugs in FFmpegs (codecs) that you can help fix or add addition information/samples to.
- Category:Formats missing in FFmpeg for formats not implemented in FFmpeg yet