Notes on first meeting » History » Version 2

« Previous - Version 2/32 (diff) - Next » - Current version
Gyorgy Fazekas, 2012-02-22 11:19 PM


Notes on first meeting

Topics discussed (roughly)

  • What are the main research use cases for an Audio Features Ontology (AF) ?
  • Are they served well by the existing AF ?
  • If not, what are the most important extensions we need to do?
  • Does the fundamental structure of the ontology need to be changed?
  • Personal Objectives: what are we going to do with a modified/re-engineered ontology?
  • What is the relation of AF to existing software, including:

    *software like Sonic Annotator, SV, SAWA, AudioDB other tools...
    *and projects like OMRAS2, EASAIER, new Semantic Media/Semantic Audio grants...

Use cases:

++Thomas:

  • drive audio effects -> adaptive effect (controlling effects)
  • KM like use case: association of audio effects and audio features e.g. pitch shifter won’t change onsets
  • part of the AFX ontology
  • more audio features
  • technical classification of audio effects
++Steve:
  • Finding structure, repeated sequences of features
  • Beat related stuff, BPM (tempo, major/minor is it an audio feature, not necessarilty)
  • Chords => Chord ontology
  • Melody and notes
++George:
  • Improve SAWA
  • Facilitate the development of intelligent music production systems
  • Release large content based metadata repositories in RDF
  • Re-release the MSD in RDF (??)
  • Deploy a knowledge based environment for content-based audio analysis based on the concept of the Knowledge Machine that can combine multiple modalities
  • Research reproducibility using Ontologies as a model to exchange research data.

Open issues:

Domain and scope:

  • Are Musicological concepts outside the scope of an AF ?
  • Physical features : Acoustic features, Perceptual Features, DSP type, Musical Features

Fundamental structure of existing AF:

  • Does it serve you well?
  • For example, loudness is defined as a segment in AF, and it does not fit a perceptual attribute well.
  • What depth do we want ? (both in terms of scope and the level of detail in describing a feature extraction workflow)
  • How AF relates to the DSP workflows used in extracting them?

Existing resources :

Some work related to Steve's use cases, segmentation and Ontologies:

Ideas/resources for new Ontologies:

  • Steve has worked on Acoustics related ontology
  • create an Acoustics Ontology
  • describe Musicological concepts
  • describe concepts related to cognitive and perceptual issues

Currently missing features

  • MFCC-s
  • Rythmogram
  • RMS energy
  • combined features, e.g. weighted combinations or statistical averages over features

Development issues

  • chaining, combination, weighting
  • how you associate features with arbitrary data
  • summary feature types
  • SM (similarity matrix) are they part of the ontoogy?
  • how to describe salience, can you hear it, can you perceive, is there an agreement
  • how to describe weighting, confidence
  • mood, music psychology, cognition, emotion, (perception ?)
  • provenance => music provenance
  • deprecation and versioning

Long term objectives:

Some concrete tasks that can be done as the outcome of the collaboration:

  • A version of Sonic Annotator that produces output adhering the new ontology
  • Are we making people happier by doing so?
  • gradual transition period?
  • extend other software toolkits; e.g. a verison of Marsyas in C++
  • multitrack processing using Sonic Annotator (this feature might come along soon)

Some immediate tasks (TODO):

  • collect more resources
  • Verify the relationship between AF as is, and other feature/segmentation Ontologies
  • what other software uses it?
  • papers and literature review
  • relation to projects e.g. SIEMAC
  • collect features that we need
  • define scope (extend the diagram of the set of ontologies: )
  • collect specific application examples from existing processing chain / workflow

collect software/projects that use/produce audio features:

  • plugins, LADSPA, VAMP, Marsyas, CLAM, libextract, COMirva, MIRtoolbox, Supercollider, other frameworks