Audio Features Ontology revision (old)

h1. Notes on first meeting

h2. Topics discussed (roughly)

** What are the main research use cases for an Audio Features Ontology (AF) ?
** Are they served well by the existing AF ?
** If not, what are the most important extensions we need to do?
** Does the fundamental structure of the ontology need to be changed?
** Personal Objectives: what are we going to do with a modified/re-engineered ontology?
** What is the relation of AF to existing software, including:

*software like Sonic Annotator, SV, SAWA, AudioDB other tools...
*and projects like OMRAS2, EASAIER, new Semantic Media/Semantic Audio grants...

h1. h2. Use cases:

++Thomas:

** drive audio effects -> adaptive effect (controlling effects)
** KM like use case: association of audio effects and audio features e.g. pitch shifter won’t change onsets
** part of the AFX ontology
** more audio features
** technical classification of audio effects

++Steve:
** Finding structure, repeated sequences of features
** Beat related stuff, BPM (tempo, major/minor is it an audio feature, not necessarilty)
** Chords => Chord ontology
** Melody and notes

++George:
** Improve SAWA
** Facilitate the development of intelligent music production systems
** Release large content based metadata repositories in RDF
** Re-release the MSD in RDF (??)
** Deploy a knowledge based environment for content-based audio analysis based on the concept of the Knowledge Machine that can combine multiple modalities
** Research reproducibility using Ontologies as a model to exchange research data.

h2. Open issues:

h3. Domain and scope:

** Are Musicological concepts outside the scope of an AF ?
** Physical features : Acoustic features, Perceptual Features, DSP type, Musical Features

h3. Fundamental structure of existing AF:

** Does it serve you well?
** For example, loudness is defined as a segment in AF, and it does not fit a perceptual attribute well.
** What depth do we want ? (both in terms of scope and the level of detail in describing a feature extraction workflow)
** How AF relates to the DSP workflows used in extracting them?

h2. Existing resources :

h3. Some work related to Steve's use cases, segmentation and Ontologies:

** SALAMI Project: Kevin Page, DaveDeRoure http://salami.music.mcgill.ca/
** The Segment Ontology: http://users.ox.ac.uk/~oerc0033/preprints/admire2011.pdf
** PopStructure Ontology: Kurt Jacobson Unpublished.
(Example available: http://wiki.musicontology.com/index.php/Structural_annotations_of_%22Can%27t_buy_me_love%22_by_the_Beatles)
** Similarity Ontology: Kurt Jacobson http://grasstunes.net/ontology/musim/musim.html

h2. Ideas/resources for new Ontologies:

** Steve has worked on Acoustics related ontology

** Creating a DSP ontology:
** include processing steps down to math operations
(this can take advantage to the math:namespace in CWM: http://www.w3.org/DesignIssues/Notation3.html)
** describe common DSP parameters

** create an Acoustics Ontology
** describe Musicological concepts
** describe concepts related to cognitive and perceptual issues

h2. Currently missing features

** MFCC-s
** Rythmogram
** RMS energy
** combined features, e.g. weighted combinations or statistical averages over features

h2. Development issues

** chaining, combination, weighting
** how you associate features with arbitrary data
** summary feature types
** SM (similarity matrix) are they part of the ontoogy?
** how to describe salience, can you hear it, can you perceive, is there an agreement
** how to describe weighting, confidence
** mood, music psychology, cognition, emotion, (perception ?)
** provenance => music provenance
** deprecation and versioning

h2. Long term objectives:

Some concrete tasks that can be done as the outcome of the collaboration:

** A version of Sonic Annotator that produces output adhering the new ontology
** Are we making people happier by doing so?
** gradual transition period?
** extend other software toolkits; e.g. a verison of Marsyas in C++
** multitrack processing using Sonic Annotator (this feature might come along soon)

h2. Some immediate tasks (TODO):

** collect more resources
** Verify the relationship between AF as is, and other feature/segmentation Ontologies
** what other software uses it?
** papers and literature review
** relation to projects e.g. SIEMAC
** collect features that we need
** define scope (extend the diagram of the set of ontologies: )
** collect specific application examples from existing processing chain / workflow

collect software/projects that use/produce audio features:

** plugins, LADSPA, VAMP, Marsyas, CLAM, libextract, COMirva, MIRtoolbox, Supercollider, other frameworks

!http://www.isophonics.net/sites/isophonics.net/files/combined-frameworks.png!