Topics discussed (roughly)¶

• What are the main research use cases for an Audio Features Ontology (AF) ?
• Are they served well by the existing AF ?
• If not, what are the most important extensions we need to do?
• Does the fundamental structure of the ontology need to be changed?
• Personal Objectives: what are we going to do with a modified/re-engineered ontology?
• What is the relation of AF to existing software, including:

  *software like Sonic Annotator, SV, SAWA, AudioDB other tools...
  *and projects like OMRAS2, EASAIER, new Semantic Media/Semantic Audio grants...

Use cases:¶

++Thomas:

• drive audio effects -> adaptive effect (controlling effects)
• KM like use case: association of audio effects and audio features e.g. pitch shifter won’t change onsets
• part of the AFX ontology
• more audio features
• technical classification of audio effects

++Steve:

• Finding structure, repeated sequences of features
• Beat related stuff, BPM (tempo, major/minor is it an audio feature, not necessarilty)
• Chords => Chord ontology
• Melody and notes

++George:

• Improve SAWA
• Facilitate the development of intelligent music production systems
• Release large content based metadata repositories in RDF
• Re-release the MSD in RDF (??)
• Deploy a knowledge based environment for content-based audio analysis based on the concept of the Knowledge Machine that can combine multiple modalities
• Research reproducibility using Ontologies as a model to exchange research data.

Open issues:¶

Domain and scope:¶

• Are Musicological concepts outside the scope of an AF ?
• Physical features : Acoustic features, Perceptual Features, DSP type, Musical Features

Fundamental structure of existing AF:¶

• Does it serve you well?
• For example, loudness is defined as a segment in AF, and it does not fit a perceptual attribute well.
• What depth do we want ? (both in terms of scope and the level of detail in describing a feature extraction workflow)
• How AF relates to the DSP workflows used in extracting them?

Existing resources :¶

Some work related to Steve's use cases, segmentation and Ontologies:¶

• SALAMI Project: Kevin Page, DaveDeRoure http://salami.music.mcgill.ca/
• The Segment Ontology: http://users.ox.ac.uk/~oerc0033/preprints/admire2011.pdf
• PopStructure Ontology: Kurt Jacobson Unpublished.
  (Example available: http://wiki.musicontology.com/index.php/Structural_annotations_of_%22Can%27t_buy_me_love%22_by_the_Beatles)
• Similarity Ontology: Kurt Jacobson http://grasstunes.net/ontology/musim/musim.html

Ideas/resources for new Ontologies:¶

• Steve has worked on Acoustics related ontology

• Creating a DSP ontology:
• include processing steps down to math operations
  (this can take advantage to the math:namespace in CWM: http://www.w3.org/DesignIssues/Notation3.html)
• describe common DSP parameters

• create an Acoustics Ontology
• describe Musicological concepts
• describe concepts related to cognitive and perceptual issues

Currently missing features¶

• MFCC-s
• Rythmogram
• RMS energy
• combined features, e.g. weighted combinations or statistical averages over features

Development issues¶

• chaining, combination, weighting
• how you associate features with arbitrary data
• summary feature types
• SM (similarity matrix) are they part of the ontoogy?
• how to describe salience, can you hear it, can you perceive, is there an agreement
• how to describe weighting, confidence
• mood, music psychology, cognition, emotion, (perception ?)
• provenance => music provenance
• deprecation and versioning

Long term objectives:¶

Some concrete tasks that can be done as the outcome of the collaboration:

• A version of Sonic Annotator that produces output adhering the new ontology
• Are we making people happier by doing so?
• gradual transition period?
• extend other software toolkits; e.g. a verison of Marsyas in C++
• multitrack processing using Sonic Annotator (this feature might come along soon)

Some immediate tasks (TODO):¶

• collect more resources
• Verify the relationship between AF as is, and other feature/segmentation Ontologies
• what other software uses it?
• papers and literature review
• relation to projects e.g. SIEMAC
• collect features that we need
• define scope (extend the diagram of the set of ontologies: )
• collect specific application examples from existing processing chain / workflow

collect software/projects that use/produce audio features:

• plugins, LADSPA, VAMP, Marsyas, CLAM, libextract, COMirva, MIRtoolbox, Supercollider, other frameworks