Notes on first meeting » History » Version 2

Gyorgy Fazekas, 2012-02-22 11:19 PM

1 1 Gyorgy Fazekas
h1. Notes on first meeting
2 1 Gyorgy Fazekas
3 1 Gyorgy Fazekas
h2. Topics discussed (roughly)
4 1 Gyorgy Fazekas
5 1 Gyorgy Fazekas
** What are the main research use cases for an Audio Features Ontology (AF) ?
6 2 Gyorgy Fazekas
** Are they served well by the existing AF ? 
7 1 Gyorgy Fazekas
** If not, what are the most important extensions we need to do?
8 1 Gyorgy Fazekas
** Does the fundamental structure of the ontology need to be changed?
9 1 Gyorgy Fazekas
** Personal Objectives: what are we going to do with a modified/re-engineered ontology?
10 1 Gyorgy Fazekas
** What is the relation of AF to existing software, including:
11 1 Gyorgy Fazekas
12 2 Gyorgy Fazekas
 *software like Sonic Annotator, SV, SAWA, AudioDB other tools... 
13 2 Gyorgy Fazekas
 *and projects like OMRAS2, EASAIER, new Semantic Media/Semantic Audio grants...
14 1 Gyorgy Fazekas
15 2 Gyorgy Fazekas
16 1 Gyorgy Fazekas
h2. Use cases:
17 1 Gyorgy Fazekas
18 1 Gyorgy Fazekas
++Thomas:
19 1 Gyorgy Fazekas
  
20 1 Gyorgy Fazekas
** drive audio effects -> adaptive effect (controlling effects)
21 1 Gyorgy Fazekas
** KM like use case: association of audio effects and audio features e.g. pitch shifter won’t change onsets
22 1 Gyorgy Fazekas
** part of the AFX ontology
23 1 Gyorgy Fazekas
** more audio features
24 1 Gyorgy Fazekas
** technical classification of audio effects
25 1 Gyorgy Fazekas
26 1 Gyorgy Fazekas
27 1 Gyorgy Fazekas
++Steve:
28 1 Gyorgy Fazekas
** Finding structure, repeated sequences of features
29 1 Gyorgy Fazekas
** Beat related stuff, BPM (tempo, major/minor is it an audio feature, not necessarilty)
30 1 Gyorgy Fazekas
** Chords => Chord ontology
31 1 Gyorgy Fazekas
** Melody and notes
32 1 Gyorgy Fazekas
33 1 Gyorgy Fazekas
34 1 Gyorgy Fazekas
++George:
35 1 Gyorgy Fazekas
** Improve SAWA
36 1 Gyorgy Fazekas
** Facilitate the development of intelligent music production systems
37 1 Gyorgy Fazekas
** Release large content based metadata repositories in RDF
38 1 Gyorgy Fazekas
** Re-release the MSD in RDF (??)
39 1 Gyorgy Fazekas
** Deploy a knowledge based environment for content-based audio analysis based on the concept of the Knowledge Machine that can combine multiple modalities
40 1 Gyorgy Fazekas
** Research reproducibility using Ontologies as a model to exchange research data.
41 1 Gyorgy Fazekas
42 1 Gyorgy Fazekas
43 1 Gyorgy Fazekas
h2. Open issues:
44 1 Gyorgy Fazekas
45 1 Gyorgy Fazekas
h3. Domain and scope:
46 1 Gyorgy Fazekas
47 1 Gyorgy Fazekas
** Are Musicological concepts outside the scope of an AF ?
48 1 Gyorgy Fazekas
** Physical features : Acoustic features, Perceptual Features, DSP type, Musical Features
49 1 Gyorgy Fazekas
50 2 Gyorgy Fazekas
h3. Fundamental structure of existing AF:
51 1 Gyorgy Fazekas
52 1 Gyorgy Fazekas
** Does it serve you well?
53 1 Gyorgy Fazekas
** For example, loudness is defined as a segment in AF, and it does not fit a perceptual attribute well.
54 1 Gyorgy Fazekas
** What depth do we want ? (both in terms of scope and the level of detail in describing a feature extraction workflow)
55 1 Gyorgy Fazekas
** How AF relates to the DSP workflows used in extracting them?
56 1 Gyorgy Fazekas
57 1 Gyorgy Fazekas
58 1 Gyorgy Fazekas
h2. Existing resources :
59 1 Gyorgy Fazekas
60 2 Gyorgy Fazekas
h3. Some work related to Steve's use cases, segmentation and Ontologies:
61 1 Gyorgy Fazekas
62 2 Gyorgy Fazekas
** SALAMI Project: Kevin Page, DaveDeRoure http://salami.music.mcgill.ca/
63 2 Gyorgy Fazekas
** The Segment Ontology: http://users.ox.ac.uk/~oerc0033/preprints/admire2011.pdf
64 2 Gyorgy Fazekas
** PopStructure Ontology: Kurt Jacobson Unpublished. 
65 2 Gyorgy Fazekas
(Example available: http://wiki.musicontology.com/index.php/Structural_annotations_of_%22Can%27t_buy_me_love%22_by_the_Beatles) 
66 2 Gyorgy Fazekas
** Similarity Ontology: Kurt Jacobson http://grasstunes.net/ontology/musim/musim.html
67 1 Gyorgy Fazekas
68 1 Gyorgy Fazekas
69 2 Gyorgy Fazekas
h2. Ideas/resources for new Ontologies:
70 1 Gyorgy Fazekas
71 2 Gyorgy Fazekas
** Steve has worked on Acoustics related ontology
72 1 Gyorgy Fazekas
73 2 Gyorgy Fazekas
** Creating a DSP ontology:
74 2 Gyorgy Fazekas
** include processing steps down to math operations 
75 2 Gyorgy Fazekas
  (this can take advantage to the math:namespace in CWM: http://www.w3.org/DesignIssues/Notation3.html)
76 2 Gyorgy Fazekas
** describe common DSP parameters
77 2 Gyorgy Fazekas
78 2 Gyorgy Fazekas
** create an Acoustics Ontology
79 2 Gyorgy Fazekas
** describe Musicological concepts
80 2 Gyorgy Fazekas
** describe concepts related to cognitive and perceptual issues
81 2 Gyorgy Fazekas
82 2 Gyorgy Fazekas
83 1 Gyorgy Fazekas
h2. Currently missing features
84 1 Gyorgy Fazekas
85 2 Gyorgy Fazekas
** MFCC-s
86 1 Gyorgy Fazekas
** Rythmogram
87 1 Gyorgy Fazekas
** RMS energy
88 2 Gyorgy Fazekas
** combined features, e.g. weighted combinations or statistical averages over features
89 1 Gyorgy Fazekas
90 1 Gyorgy Fazekas
91 1 Gyorgy Fazekas
h2. Development issues
92 1 Gyorgy Fazekas
93 1 Gyorgy Fazekas
** chaining, combination, weighting
94 1 Gyorgy Fazekas
** how you associate features with arbitrary data
95 2 Gyorgy Fazekas
** summary feature types 
96 1 Gyorgy Fazekas
** SM (similarity matrix) are they part of the ontoogy?
97 2 Gyorgy Fazekas
** how to describe salience, can you hear it, can you perceive, is there an agreement
98 2 Gyorgy Fazekas
** how to describe weighting, confidence
99 1 Gyorgy Fazekas
** mood, music psychology, cognition, emotion, (perception ?)
100 1 Gyorgy Fazekas
** provenance => music provenance
101 2 Gyorgy Fazekas
** deprecation and versioning
102 1 Gyorgy Fazekas
103 1 Gyorgy Fazekas
104 1 Gyorgy Fazekas
h2. Long term objectives:
105 1 Gyorgy Fazekas
106 1 Gyorgy Fazekas
Some concrete tasks that can be done as the outcome of the collaboration:
107 1 Gyorgy Fazekas
108 2 Gyorgy Fazekas
** A version of Sonic Annotator that produces output adhering the new ontology
109 1 Gyorgy Fazekas
** Are we making people happier by doing so?
110 2 Gyorgy Fazekas
** gradual transition period?
111 2 Gyorgy Fazekas
** extend other software toolkits; e.g. a verison of Marsyas in C++
112 2 Gyorgy Fazekas
** multitrack processing using Sonic Annotator (this feature might come along soon)
113 1 Gyorgy Fazekas
114 1 Gyorgy Fazekas
115 2 Gyorgy Fazekas
h2. Some immediate tasks (TODO):
116 1 Gyorgy Fazekas
117 2 Gyorgy Fazekas
** collect more resources 
118 2 Gyorgy Fazekas
** Verify the relationship between AF as is, and other feature/segmentation Ontologies
119 2 Gyorgy Fazekas
** what other software uses it?
120 2 Gyorgy Fazekas
** papers and literature review
121 2 Gyorgy Fazekas
** relation to projects e.g. SIEMAC
122 2 Gyorgy Fazekas
** collect features that we need
123 2 Gyorgy Fazekas
** define scope (extend the diagram of the set of ontologies: )
124 2 Gyorgy Fazekas
** collect specific application examples from existing processing chain / workflow
125 1 Gyorgy Fazekas
126 2 Gyorgy Fazekas
collect software/projects that use/produce audio features:
127 1 Gyorgy Fazekas
128 1 Gyorgy Fazekas
** plugins, LADSPA, VAMP, Marsyas, CLAM, libextract, COMirva, MIRtoolbox, Supercollider, other frameworks