Audio Features Ontology revision (old)

h1. Notes on first meeting

h2. Topics discussed (roughly)

** What are the main research use cases for an Audio Features Ontology (AF) ?

** Are they served by the existing AF well? 

** If not, what are the most important extensions we need to do?

** Does the fundamental structure of the ontology need to be changed?

** Personal Objectives: what are we going to do with a modified/re-engineered ontology?

** What is the relation of AF to existing software, including:

Sonic Annotator, SV, SAWA, other tools... and projects OMRAS2, EASAIER client, AudioDB

h2. Use cases:

++Thomas:

** drive audio effects -> adaptive effect (controlling effects)

** KM like use case: association of audio effects and audio features e.g. pitch shifter won’t change onsets

** part of the AFX ontology

** more audio features

** technical classification of audio effects

++Steve:

** Finding structure, repeated sequences of features

** Beat related stuff, BPM (tempo, major/minor is it an audio feature, not necessarilty)

** Chords => Chord ontology

** Melody and notes

++George:

** Improve SAWA

** Facilitate the development of intelligent music production systems

** Release large content based metadata repositories in RDF

** Re-release the MSD in RDF (??)

** Deploy a knowledge based environment for content-based audio analysis based on the concept of the Knowledge Machine that can combine multiple modalities

** Research reproducibility using Ontologies as a model to exchange research data.

h2. Resources:

h3. Some work related to Steve's use cases, segmentation and Ontologies:

** SALAMI Project: Kevin Page, DaveDeRoure http://salami.music.mcgill.ca/

** The Segment Ontology: http://users.ox.ac.uk/~oerc0033/preprints/admire2011.pdf

** PopStructure Ontology: Kurt Jacobson Unpublished. 

(Example available: http://wiki.musicontology.com/index.php/Structural_annotations_of_%22Can%27t_buy_me_love%22_by_the_Beatles) 

** Similarity Ontology: Kurt Jacobson http://grasstunes.net/ontology/musim/musim.html

h2. Open issues:

h3. Domain and scope:

** Are Musicological concepts outside the scope of an AF ?

** Physical features : Acoustic features, Perceptual Features, DSP type, Musical Features

h3. Fundamental structure:

** Does it serve you well?

** For example, loudness is defined as a segment in AF, and it does not fit a perceptual attribute well.

** What depth do we want ? (both in terms of scope and the level of detail in describing a feature extraction workflow)

** How AF relates to the DSP workflows used in extracting them?

h2. Existing resources :

Ideas for new ontologies:

Steve: has worked on Acoustics related ontology

DSP ontology 

** processing steps down to math operation

** DSP parameters

Acoustics ontology

Musicological concepts

h2. Currently missing features

** MFCC-s are missing

** Rythmogram

** RMS energy

** combined features, e.g. weighted combinations, statistical averages over features

h2. Development issues

** chaining, combination, weighting

** how you associate features with arbitrary data

** summary feature types

** SM (similarity matrix) are they part of the ontoogy?

** salience, can you hear it, can you perceive, is there an agreement

** weighting, confidence

** mood, music psychology, cognition, emotion, (perception ?)

** provenance => music provenance

** need for deprecation and versioning

h2. Long term objectives:

Some concrete tasks that can be done as the outcome of the collaboration:

** Sonic Annotator produces output according to the new ontology

** Are we making people happier by doing so?

** gradual transition.

** software toolkits; a verison of Marsyas C++

** multitrack processing using Sonic Annotator

h2. Immediate tasks (TODO):

** collect resources 

** Verify the relationship between AF as is, and other segmentation ontologies

** software

** papers

** projects

** SIEMAC

** collect features that we need.

** plugins, LADSPA, VAMP, Marsyas, CLAM, libextract, COMirva, MIRtoolbox, Supercollider, other frameworks

** scope (diagram of a set of ontologies)

** specific application examples from existing processing chain / workflow