MPEG Developments

MPEG-A: Interactive Music Application Format

MixRights (featuring collaborative music creation & remixing, karaoke & chords, tagging & sharing in social nets & counting .... music citations !) will be presented at Interactive Music Hack-Fest in London, 11 June 2016 & Sonar+D in Barcelona, 16-18 June 2016.

&

Sonic Visualiser with IM AF support featuring automatic chords extraction aligned in time with lyrics:
Executables for windows & mac os/x & Source code
La Fuga an IM AF (.ima) song by Emilio Molina Martínez.
Tiny Human an IM AF (.ima) song by Imogen Heap.

We would like to further enhance IM AF with new technologies. Topics for student projects available (at all levels UG/MSc/PhD) to be taken are given below. Programming languages background is a prerequisite as well as willingness to learn new ones, e.g., HTML5. Furthermore, students are expected to develop knowledge and skills on working with latest standards (MPEG, EBU, DMP) as well as acquire commercial and market perspective of music technologies for internet, broadcasting and mobile applications.

• Lyrics extraction and/or highlighting for karaoke apps EPSRC: Music Repurposing
• Rights tracking for fair artists remuneration MVCO (Rethink Music & Mycelia)
• Dynamic adaptive audio streaming DASH
• 3D audio mixing MPEG-H Audio
• Second screen music recommendation
• Vocaloid animation

If you would like to involve in this project please contact p.kudumakis at qmul.ac.uk

Contributors

• Thomas Wilmering (C4DM Post-doc)
  Music remix: Derivative works rights tracking
  Requirements for a Media Value Chain Ontology
  An example of rights tracking by ISO/IEC 21000-19 AMD 1: MVCO extensions on time segments & multi-track audio, June 2016

• Mariano Mora-Mcginity (C4DM PhD Candidate)
  HTML5 IM AF Player ver. 3
  Featuring music remix
  DMRN+10 Dec. 2015

• Luke Barlow (C4DM collaborator)
  HTML5 IM AF Player ver. 2
  Featuring robustness & zoom in/out
  DMRN+9 Dec. 2014

• Jesús C. García (Univ. of Malaga)
  IM AF codec in Sonic Visualiser
  Featuring lyrics, metadata, dynamic volume, URL
  MSc Thesis, Jun. 2014 - in Spanish Awarded Best Thesis by COIT-AORM & AITA, 30 Oct. 2015
  53rd AES Paper, Jan. 2014

• Costantino Taglialatela (C4DM visitor)
  IM AF codec in Sonic Visualiser
  Featuring still pictures, rules, presets, groups, EQ
  MSc Thesis, Jul. 2013
  53rd AES Paper, Jan. 2014

• Giacomo Herrero (Univ. of Malaga)
  HTML5 IM AF Player ver. 1
  Featuring multi-track audio, lyrics, album picture & metadata
  CMMR'13 Paper, Oct. 2013

• Eugenio Oñate (C4DM)
  IM AF encoder: proof of concept
  Featuring multi-track audio
  MSc Thesis, Aug. 2012
  DMRN+7 Paper, Dec. 2012

Related publications

• Jesús C. García, Panos Kudumakis, Isabel Barbancho, Lorenzo J. Tardón and Mark Sandler, 'Enabling interactive and interoperable semantic music applications', Springer Handbook for Systematic Musicology, Apr. 2014. In press.

• Jesús C. García, Costantino Taglialatela, Panos Kudumakis, Lorenzo J. Tardón, Isabel Barbancho and Mark Sandler, 'Interactive music applications by MPEG-A support in Sonic Visualiser', AES 53rd International Conference on Semantic Audio, London, UK, 27-29 Jan. 2014.

• Giacomo Herrero, Panos Kudumakis, Lorenzo J. Tardon, Isabel Barbancho, Mark Sandler, 'An HTML5 Interactive (MPEG-A IM AF) Music Player', 10th International Symposium on Computer Music Multidisciplinary Research (CMMR'13), Marseille, France, 15-18 Oct. 2013.

• Rebecca Stewart, Panos Kudumakis, and Mark Sandler, 'Interactive Music Applications and Standards', in 'Exploring Music Contents', Lecture Notes in Computer Science, pp. 20-30, Vol. 6684, Springer Berlin Heidelberg (ISBN 978-3-642-23125-4), August 2011. Editors: S. Ystad, M. Aramaki, R. Kronland-Martinet & K. Jensen.

• Panos Kudumakis, 'MP3: something's gotta change!', in 'Audio! Magazine', pp. 6, Vol. 1, Issue 3, April 2011. Editors: Paul Curzon, Mathieu Barthet and Simon Dixon.

• Inseon Jang, Panos Kudumakis, Mark Sandler, Kyeongok Kang, 'The MPEG Interactive Music Application Format Standard', IEEE Signal Processing Magazine, pp. 150-154, Vol. 28, Issue 1, Jan. 2011.

• Panos Kudumakis, Inseon Jang and Mark Sandler, 'A New Interactive MPEG Format for the Music Industry', 7th International Symposium on Computer Music Modeling and Retrieval (CMMR'10), Málaga, Spain, 21-24 June, 2010.

Inspirations

• IM AF: What's next?
• IM AF MSc study by Tracy Redhead - 2015
• IM AF functionality in STEMS by Native Instruments - 2015
• IM AF/STEMS to be supported in LANDR by MixGenius - 2016
• IM AF/(STEMS soon) tablet DJ mixing with MixRights & Tuna Knobs - 2016