MPEG Developments

!>https://code.soundsoftware.ac.uk/attachments/download/807/Malaga_z3770.jpg(Invited talk on IM AF by Panos Kudumakis, Malaga, Spain 11 April 2013)!

*MPEG-A: Interactive Music Application Format*

>"MixRights":http://mixrights.eecs.qmul.ac.uk/ featuring collaborative music creation & remixing, karaoke & chords, tagging & sharing in social nets & counting .... music citations! "On YouTube":https://youtu.be/Kce_Jolsbro
>
>*Events*
>"IEEE UK & Ireland Members Open Day 2017,":https://westminster.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=daacfdd9-8168-4694-b12a-297e8a5ce303 London, 7 Oct. 2017 (#t=4h17m)
>"Interactive Music Summit,":https://www.eventbrite.com/e/limm-interactive-music-summit-tickets-28533269778 Google Campus, London, 25 Nov. 2016
>"Sonar+D,":http://sonarplusd.com/activity/queen-mary-university-of-london/ Barcelona, 16-18 June 2016.
>"Interactive Music Hack-Fest,":https://www.eventbrite.co.uk/e/interactive-music-hack-fest-tickets-25513856632 London, 11 June 2016

p=. &

>Sonic Visualiser with IM AF support featuring automatic chords extraction aligned in time with lyrics:
>"Executables":https://code.soundsoftware.ac.uk/projects/sv-imaf-enc/files for windows & mac os/x & "Source code":https://code.soundsoftware.ac.uk/projects/sv-imaf-enc
>"La Fuga":https://code.soundsoftware.ac.uk/attachments/download/871/La%20fuga%20de%20logan.ima an IM AF (.ima) song by Emilio Molina Martínez.
>"Tiny Human":https://code.soundsoftware.ac.uk/attachments/download/1800/1TinyHuman.ima an IM AF (.ima) song by "Imogen Heap":http://imogenheap.com/home.php?article=2430.

We would like to further enhance IM AF with new technologies. Topics for student projects available (at all levels UG/MSc/PhD) to be taken are given below. Programming languages background is a prerequisite as well as willingness to learn new ones, e.g., HTML5. Furthermore, students are expected to develop knowledge and skills on working with latest standards (MPEG, EBU, DMP) as well as acquire commercial and market perspective of music technologies for internet, broadcasting and mobile applications.
* Lyrics extraction and/or highlighting for karaoke apps "EPSRC: Music Repurposing":http://gow.epsrc.ac.uk/NGBOViewGrant.aspx?GrantRef=EP/L027119/2
* Rights tracking for fair artists remuneration "MPEG IPR Ontologies":https://code.soundsoftware.ac.uk/attachments/download/2481/1PanosGeneva126MPEGMeetingReport.pdf / "AVCO":https://www.linkedin.com/pulse/smart-contracts-fair-trade-music-panos-kudumakis/ ("Rethink Music":http://www.rethink-music.com/press/berklee-institute-for-creative-entrepreneurships-rethink-music-releases-report-on-transparency-and-fairness-in-the-music-industry-1 & "Mycelia":http://myceliaformusic.org/)
* Dynamic adaptive audio streaming "MPEG-DASH":http://dashif.org/
* 3D audio mixing "MPEG-H Audio":http://www.mpeghaa.com/index.html
* Second screen music recommendation
* Vocaloid animation

If you would like to involve in this project please contact p.kudumakis at qmul.ac.uk

*Contributors*

* Thomas Wilmering (C4DM Post-doc)
Music remix & derivative works IP rights tracking with
*MPEG-21 - Part 19: Media Value Chain Ontology (MVCO) AMENDMENT 1: Extensions on Time-Segments & Multi-Track Audio (AVCO)*
"Text of ISO/IEC 21000-19 AMD 1":https://www.iso.org/standard/71978.html (June 2018) extends "ISO/IEC 21000-19":https://www.iso.org/standard/52887.html
*MPEG-21 - Part 8: Reference Software AMENDMENT 4: MVCO Extensions on Time-Segments & Multi-Track Audio (AVCO)*
"Reference Software ISO/IEC 21000-8 AMD 4":https://www.iso.org/standard/74432.html (Oct 2018) extends "ISO/IEC 21000-8 AMD 2":https://www.iso.org/standard/57394.html
"Source Code & Guidelines ISO/IEC 21000-8 AMD 4":http://standards.iso.org/iso-iec/21000/-8/ed-2/en/amd/4 (Sept 2018) extends "ISO/IEC 21000-8 AMD 2":http://standards.iso.org/ittf/PubliclyAvailableStandards/c057394_ISO_IEC_21000-8_2008_Amd_2_2011_Reference_Software.zip
"Audio Value Chain Ontology":https://code.soundsoftware.ac.uk/attachments/download/2396/aumvco.owl (Sept 2018) extends "Media Value Chain Ontology":http://purl.oclc.org/NET/mvco.owl

* Mariano Mora-Mcginity (C4DM PhD Candidate)
"HTML5 IM AF Player ver. 3":http://multitrack.eecs.qmul.ac.uk/imaf/player.html
Featuring music remix
DMRN+10 Dec. 2015

* Luke Barlow (C4DM collaborator)
"HTML5 IM AF Player ver. 2":http://forkaudio.org/wemix/imaf-rc.html
Featuring robustness & zoom in/out
DMRN+9 Dec. 2014

* Jesús C. García (Univ. of Malaga)
IM AF codec in Sonic Visualiser
Featuring lyrics, metadata, dynamic volume, URL
"MSc Thesis, Jun. 2014 - in Spanish":http://hdl.handle.net/10630/7699 "Awarded Best Thesis":http://www.coit-aorm.org/images/galeries/SanGabrielMalaga2015/Foto1.JPG by "COIT-AORM & AITA":http://www.coit-aorm.org/index.php/actualidad/noticias/187-celebracion-san-gabriel-malaga, 30 Oct. 2015
"53rd AES Paper, Jan. 2014":https://code.soundsoftware.ac.uk/attachments/download/972/53AESPaper.pdf

* Costantino Taglialatela (C4DM visitor)
IM AF codec in Sonic Visualiser
Featuring still pictures, rules, presets, groups, EQ
"MSc Thesis, Jul. 2013":https://code.soundsoftware.ac.uk/attachments/download/801/TESI_TAGLIALATELA.pdf
"53rd AES Paper, Jan. 2014":https://code.soundsoftware.ac.uk/attachments/download/972/53AESPaper.pdf

* Giacomo Herrero (Univ. of Malaga)
"HTML5 IM AF Player ver. 1":http://www.giacomoherrero.com/clientside/
Featuring multi-track audio, lyrics, album picture & metadata
"CMMR'13 Paper, Oct. 2013":https://code.soundsoftware.ac.uk/attachments/download/802/CMMR13.pdf

* Eugenio Oñate (C4DM)
IM AF encoder: proof of concept
Featuring multi-track audio
"MSc Thesis, Aug. 2012":https://code.soundsoftware.ac.uk/projects/enc-imaf/repository/raw/IM_AF%20Report/Development%20of%20an%20IM%20AF%20Encoder.pdf
"DMRN+7 Paper, Dec. 2012":https://code.soundsoftware.ac.uk/projects/enc-imaf/repository/raw/IM_AF%20Report/IMAF%20Paper.pdf

*Related publications*

* "MPEG Requirements Group, *'N19504 - CfP on technologies for MPEG-21 IPR contracts to smart contracts conversion & N19505 - MPEG-21 IPR contracts in XML, RDF and JSON for OMI use cases'*, 131st MPEG Meeting, Geneva (CH) - Virtual, 29 June - 03 July 2020.":https://code.soundsoftware.ac.uk/attachments/download/2736/131MPEG%20-%20CfP%20on%20smart%20contracts%20and%20related%20test%20data.zip

* "Philip Merill, *'Intellectual Property Metadata for Automated Contract Support'*, The Moving Picture Experts Group, May 2020.":https://mpeg.chiariglione.org/news/intellectual-property-metadata-automated-contract-support

* "Panos Kudumakis, Thomas Wilmering, Mark Sandler, Víctor Rodríguez-Doncel, Laurent Boch, Jaime Delgado, *'The Challenge: From MPEG Intellectual Property Rights Ontologies to Smart Contracts and Blockchains'*, IEEE Signal Processing Magazine, pp. 89-95, Vol. 37, Issue 2, Mar. 2020.":http://doi.org/10.1109/MSP.2019.2955207

* "Panos Kudumakis, Thomas Wilmering, Mark Sandler, Jeremy Foss, *'MPEG IPR Ontologies for Media Trading and Personalization'*, Proceedings of the 1st International Workshop on Data-Driven Personalization of Television (DataTV’19) co-located with the ACM International Conference on Interactive Experiences for Television and Online Video (TVX’19), Manchester, UK, 5-7 Jun. 2019.":http://ceur-ws.org/Vol-2423/DataTV2019_paper_2.pdf

* "Panos Kudumakis, Thomas Wilmering, Mark Sandler, Víctor Rodríguez-Doncel, Laurent Boch, Jaime Delgado, *'MPEG Intellectual Property Rights Ontologies'*, Ref. as ISO/IEC JTC1/SC29/WG11/N18500, Geneva, Switzerland, Mar. 2019.":https://mpeg.chiariglione.org/sites/default/files/files/standards/docs/w18500.zip

* "Panos Kudumakis and Thomas Wilmering, *'Smart Contracts for Fair Trade of Music'*, FAST IMPACTt Blog Article, 12 Oct. 2018.":http://www.semanticaudio.ac.uk/blog/smart-contracts-for-fair-trade-of-music/

* "Jesús C. García, Panos Kudumakis, Isabel Barbancho, Lorenzo J. Tardón and Mark Sandler, *'Enabling interactive and interoperable semantic music applications'*, in 'Music and Media', Springer Handbook for Systematic Musicology, pp. 911-921, Ch.45, Springer Berlin Heidelberg, Jan. 2018. Editor: R. Bader.":https://dx.doi.org/10.1007/978-3-662-55004-5_45

* "Panos Kudumakis, *'To Blockchain, or Not Blockchain?'*, FAST IMPACTt Blog Article, 20 July 2016.":http://www.semanticaudio.ac.uk/blog/blockchain-or-not-blockchain/

* "Jesús C. García, Costantino Taglialatela, Panos Kudumakis, Lorenzo J. Tardón, Isabel Barbancho and Mark Sandler, *'Interactive music applications by MPEG-A support in Sonic Visualiser'*, AES 53rd International Conference on Semantic Audio, London, UK, 27-29 Jan. 2014.":https://code.soundsoftware.ac.uk/attachments/download/972/53AESPaper.pdf

* "Giacomo Herrero, Panos Kudumakis, Lorenzo J. Tardon, Isabel Barbancho and Mark Sandler, *'An HTML5 Interactive (MPEG-A IM AF) Music Player'*, 10th International Symposium on Computer Music Multidisciplinary Research (CMMR'13), Marseille, France, 15-18 Oct. 2013.":https://code.soundsoftware.ac.uk/attachments/download/802/CMMR13.pdf

* "Rebecca Stewart, Panos Kudumakis, and Mark Sandler, *'Interactive Music Applications and Standards'*, in 'Exploring Music Contents', Lecture Notes in Computer Science, pp. 20-30, Vol. 6684, Springer Berlin Heidelberg (ISBN 978-3-642-23125-4), August 2011. Editors: S. Ystad, M. Aramaki, R. Kronland-Martinet & K. Jensen.":https://books.google.co.uk/books?id=LZr-BDXW7RUC&pg=PA20

* "Panos Kudumakis, *'MP3: something's gotta change!'*, in 'Audio! Magazine', pp. 6, Vol. 1, Issue 3, April 2011. Editors: Paul Curzon, Mathieu Barthet and Simon Dixon.":https://cs4fndownloads.files.wordpress.com/2016/03/audioissue3.pdf#page=6

* "Inseon Jang, Panos Kudumakis, Mark Sandler and Kyeongok Kang, *'The MPEG Interactive Music Application Format Standard'*, IEEE Signal Processing Magazine, pp. 150-154, Vol. 28, Issue 1, Jan. 2011.":https://code.soundsoftware.ac.uk/attachments/download/816/IEEESPMagazineJan2011.pdf

* Panos Kudumakis, Inseon Jang and Mark Sandler, *'A New Interactive MPEG Format for the Music Industry'*, 7th International Symposium on Computer Music Modeling and Retrieval (CMMR'10), Málaga, Spain, 21-24 June, 2010.

*Inspirations*

* "IM AF: What's next?":http://www.dmpf.org/o-brainstorming/PanosDMP11Jan2014.pdf#page=4
* IM AF "MSc study":http://eprints.qut.edu.au/80881/ by "Tracy Redhead":http://www.tracyredhead.com/?q=content/interactive-music-application-format-0 - 2015
* IM AF "commercialised":http://ismir2015.uma.es/LBD/LBD39.pdf as "STEMS by Native Instruments":https://www.stems-music.com/ - 2015
* IM AF/STEMS "mastering":https://www.youtube.com/watch?v=K5CsEUIh_GI with "LANDR by MixGenius":https://en.wikipedia.org/wiki/LANDR - 2017
* IM AF/STEMS tablet DJ mixing with "MixRights":http://mixrights.eecs.qmul.ac.uk/ & "Tuna Knobs":https://www.pepperdecks.com/products/28-tunagear