Notes on first meeting¶

Topics discussed (roughly)¶

• What are the main research use cases for an Audio Features Ontology (AF) ?
• Are they served by the existing AF well?
• If not, what are the most important extensions we need to do?
• Does the fundamental structure of the ontology need to be changed?
• Personal Objectives: what are we going to do with a modified/re-engineered ontology?
• What is the relation of AF to existing software, including:
  Sonic Annotator, SV, SAWA, other tools... and projects OMRAS2, EASAIER client, AudioDB

Use cases:¶

++Thomas:

• drive audio effects -> adaptive effect (controlling effects)
• KM like use case: association of audio effects and audio features e.g. pitch shifter won’t change onsets
• part of the AFX ontology
• more audio features
• technical classification of audio effects

++Steve:

• Finding structure, repeated sequences of features
• Beat related stuff, BPM (tempo, major/minor is it an audio feature, not necessarilty)
• Chords => Chord ontology
• Melody and notes

++George:

• Improve SAWA
• Facilitate the development of intelligent music production systems
• Release large content based metadata repositories in RDF
• Re-release the MSD in RDF (??)
• Deploy a knowledge based environment for content-based audio analysis based on the concept of the Knowledge Machine that can combine multiple modalities
• Research reproducibility using Ontologies as a model to exchange research data.

Resources:¶

Some work related to Steve's use cases, segmentation and Ontologies:¶

• SALAMI Project: Kevin Page, DaveDeRoure http://salami.music.mcgill.ca/
• The Segment Ontology: http://users.ox.ac.uk/~oerc0033/preprints/admire2011.pdf
• PopStructure Ontology: Kurt Jacobson Unpublished.
  (Example available: http://wiki.musicontology.com/index.php/Structural_annotations_of_%22Can%27t_buy_me_love%22_by_the_Beatles)
• Similarity Ontology: Kurt Jacobson http://grasstunes.net/ontology/musim/musim.html

Open issues:¶

Domain and scope:¶

• Are Musicological concepts outside the scope of an AF ?
• Physical features : Acoustic features, Perceptual Features, DSP type, Musical Features

Fundamental structure:¶

• Does it serve you well?
• For example, loudness is defined as a segment in AF, and it does not fit a perceptual attribute well.
• What depth do we want ? (both in terms of scope and the level of detail in describing a feature extraction workflow)
• How AF relates to the DSP workflows used in extracting them?

Existing resources :¶

Ideas for new ontologies:

Steve: has worked on Acoustics related ontology

DSP ontology

• processing steps down to math operation
• DSP parameters

Acoustics ontology
Musicological concepts

Currently missing features¶

• MFCC-s are missing
• Rythmogram
• RMS energy
• combined features, e.g. weighted combinations, statistical averages over features

Development issues¶

• chaining, combination, weighting
• how you associate features with arbitrary data

• summary feature types

• SM (similarity matrix) are they part of the ontoogy?

• salience, can you hear it, can you perceive, is there an agreement
• weighting, confidence

• mood, music psychology, cognition, emotion, (perception ?)

• provenance => music provenance

• need for deprecation and versioning

Long term objectives:¶

Some concrete tasks that can be done as the outcome of the collaboration:

• Sonic Annotator produces output according to the new ontology

• Are we making people happier by doing so?
• gradual transition.

• software toolkits; a verison of Marsyas C++

• multitrack processing using Sonic Annotator

Immediate tasks (TODO):¶

• collect resources
• Verify the relationship between AF as is, and other segmentation ontologies
• software
• papers
• projects
• SIEMAC

• collect features that we need.

• plugins, LADSPA, VAMP, Marsyas, CLAM, libextract, COMirva, MIRtoolbox, Supercollider, other frameworks

• scope (diagram of a set of ontologies)

• specific application examples from existing processing chain / workflow