Wiki » History » Version 5

Mi Tian, 2015-08-05 09:07 PM

1 1 Mi Tian
h1. Wiki
2 1 Mi Tian
3 1 Mi Tian
h2. Vamp environment for feature extraction
4 1 Mi Tian
5 4 Mi Tian
_Vamp_ is an audio processing plugin system for plugins that extract descriptive information from audio data — typically referred to as audio analysis plugins or audio feature extraction plugins. The _vamp plugins_ are distributed in shared library files with extension _.dll_, _.so_, or _.dylib_ depending on the platform.  
6 1 Mi Tian
7 4 Mi Tian
Several common audio file formats are supported, including MP3, Ogg, and a number of PCM formats such as WAV and AIFF. AAC is supported on OS/X only, and only if not DRM protected. WMA is not supported.
8 1 Mi Tian
9 5 Mi Tian
See [[http://vamp-plugins.org/]] for more introduction about Vamp environment and the plugins.
10 5 Mi Tian
11 1 Mi Tian
h2. Host applications
12 1 Mi Tian
13 4 Mi Tian
A vamp plugin cannot be used on its own, but only with a conforming _host_ application which loads the plugin from its shared library and calls functions within the plugin code to configure it, supply it with data, and run it.
14 4 Mi Tian
15 4 Mi Tian
* Sonic Visualiser -- An interactive audio analysis and visualisation tool using Vamp plugins
16 1 Mi Tian
[[http://www.sonicvisualiser.org/]]
17 4 Mi Tian
* Sonic Annotator -- A command line tool facilitating facilitating feature extraction, parameter configuration and output manipulation for a large bunch of audio files.
18 1 Mi Tian
[[https://code.soundsoftware.ac.uk/projects/sonic-annotator]]
19 4 Mi Tian
* Audacity -- An audio editor which enables recording, labelling and editing the audio.
20 1 Mi Tian
[[http://audacityteam.org/]]
21 1 Mi Tian
22 4 Mi Tian
h2. Plugin installation
23 1 Mi Tian
24 1 Mi Tian
The vamp plugins need to be put in a directory where the host can locate, i.e., the _VAMP_PATH_. See [[http://vamp-plugins.org/download.html#install]] for installation instructions. 
25 1 Mi Tian
26 5 Mi Tian
Alternatively, you can set your own VAMP_PATH environment variable to list the locations a host should look in for Vamp plugins. For example, the following command will set @/homes/mitian/hg/onset-fusion@ as an alternative VAMP_PATH. Plugins placed in this directory will also be locatable for the hosts besides those under the system specified vamp directories.
27 4 Mi Tian
<pre>
28 5 Mi Tian
mitian$ export VAMP_PATH=$VAMP_PATH:/homes/mitian/hg/onset-fusion
29 4 Mi Tian
</pre>
30 4 Mi Tian
31 4 Mi Tian
Once the plugins are installed in the Vamp path, they will show themselves under the _Transform_ tab in Sonic Visualiser, or the _Analyze_ tab in Audacity, or in the terminal when you list them using Sonic Annotator (@ sonic-annotator -l@). 
32 4 Mi Tian
33 1 Mi Tian
h2. Parameters and outputs of this plugin set
34 4 Mi Tian
35 4 Mi Tian
h3. Parameters
36 4 Mi Tian
<pre>
37 5 Mi Tian
    **Adjustable parameters**
38 4 Mi Tian
	Fusion Type 					   Fusion type [0: early fusion, 1: linear fusion, 2: decision fusion]
39 4 Mi Tian
	Detection Sensitivity   	    Sensitivity level for peak detection in the onset likelihood function.
40 4 Mi Tian
								       The higher the sensitivity, the more onsets will (rightly or wrongly) be detected.
41 4 Mi Tian
	Delta Threshold  			   Constant offset used for adaptive theresholding using the median of the detection function.
42 4 Mi Tian
	Backtracking Threshold  		   Backtracking threshold used determine the stopping condition for backtracking the detected onset location
43 4 Mi Tian
									to an earlier 'perceived' location.
44 1 Mi Tian
	Tolerance window				   Tolerance window to combine events from different detection functions for decision fusion.
45 4 Mi Tian
	Linear combination coefficient	 Combination coefficient for linear fusion.
46 4 Mi Tian
	Median filter window			   Median filter window size.
47 4 Mi Tian
	Cutoff frequency				   Cutoff frequency for the low pass filter.
48 4 Mi Tian
	
49 5 Mi Tian
	**bool parameters**		      [0: off, 1: on]
50 4 Mi Tian
	Polynomial fitting				 Use polynomial fitting to evaluate detection function peaks.
51 4 Mi Tian
	Median filtering				   Use median filter based adaptive thresholding.
52 4 Mi Tian
	Low-pass filtering				 Use low pass filtering to smooth onset detection functions.
53 4 Mi Tian
	Adaptive whitening				 Use adaptive whitening. It evens out the temporal and frequency variation in the signal.
54 4 Mi Tian
55 4 Mi Tian
</pre>
56 4 Mi Tian
57 4 Mi Tian
h3. Outputs
58 4 Mi Tian
59 1 Mi Tian
60 4 Mi Tian
# Note Onsets – The detected note onset times, returned as a single feature with timestamp.
61 4 Mi Tian
# Smoothed Detection Function – The smoothed onset detection function.
62 1 Mi Tian
# Note Onsets – The detected note onset times, returned as a single feature with timestamp.
63 1 Mi Tian
64 1 Mi Tian
65 5 Mi Tian
NOTE: The last two outputs are not available in fusion based onset detectors. Please check individual plugins for specific output types.
66 1 Mi Tian
 
67 1 Mi Tian
h2. Example usages of the onset detectors
68 5 Mi Tian
69 5 Mi Tian
To use these onset detector plugins in Sonic Visualiser and Audacity is quite straightforward. Load your audio file first, then select a feature extractor/plugin from the transform list. You will also find a set of adjustable parameters in the pop-out box associated with each transform. The extracted features, whether it is onset locations or detection functions, will be visualised in the pane on top of the waveform, synchronised with time.
70 5 Mi Tian
 
71 5 Mi Tian
For comprehensive user manuals, see
72 5 Mi Tian
[[http://www.sonicvisualiser.org/doc/reference/2.1/en/index.html]]  
73 5 Mi Tian
[[http://wiki.audacityteam.org/wiki/Category:Tutorial]]
74 3 Mi Tian
75 3 Mi Tian
<pre>
76 3 Mi Tian
mitian$ sonic-annotator -l | grep vampy-onsets
77 3 Mi Tian
vamp:vampy:vampy-onsets-bersf:vampy-onsets-bersf-onsets
78 3 Mi Tian
vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
79 3 Mi Tian
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-df
80 3 Mi Tian
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-onsets
81 3 Mi Tian
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-sdf
82 3 Mi Tian
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-df
83 3 Mi Tian
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-onsets
84 3 Mi Tian
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-sdf
85 3 Mi Tian
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-df
86 3 Mi Tian
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-onsets
87 3 Mi Tian
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-sdf
88 3 Mi Tian
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-df
89 3 Mi Tian
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-onsets
90 3 Mi Tian
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-sdf
91 1 Mi Tian
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-df
92 3 Mi Tian
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-onsets
93 3 Mi Tian
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-sdf
94 3 Mi Tian
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-df
95 3 Mi Tian
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-onsets
96 3 Mi Tian
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-sdf
97 3 Mi Tian
mitian$
98 4 Mi Tian
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
99 3 Mi Tian
Extracting and writing features... Done
100 3 Mi Tian
101 1 Mi Tian
Vampy::~PyExtensionManager: Extension module cleaned.
102 3 Mi Tian
mitian$
103 3 Mi Tian
mitian$
104 3 Mi Tian
</pre>
105 3 Mi Tian
106 3 Mi Tian
Sonic Anntator also allows for batch processing in one go. 
107 3 Mi Tian
<pre>
108 1 Mi Tian
mitian$
109 4 Mi Tian
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv /homes/mitian/documents/audio/*.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
110 1 Mi Tian
Extracting and writing features... Done
111 3 Mi Tian
112 3 Mi Tian
Vampy::~PyExtensionManager: Extension module cleaned.
113 3 Mi Tian
mitian$
114 3 Mi Tian
mitian$
115 3 Mi Tian
</pre>
116 3 Mi Tian
117 4 Mi Tian
h2. Play around with your vamp plugin feature extractors
118 3 Mi Tian
119 4 Mi Tian
We may want use parameter settings other the default. With Sonic Visualiser or Audacity, you can easily access the GUI to modify them. Whiles with Sonic Annotator, you would need to interact with 
120 3 Mi Tian
121 4 Mi Tian
Firstly, we can check the current parameter configurations in the skeleton transform description using Sonic Annotator's @-s@ command.
122 3 Mi Tian
123 3 Mi Tian
<pre>
124 3 Mi Tian
mitian$
125 3 Mi Tian
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
126 3 Mi Tian
@prefix xsd:      <http://www.w3.org/2001/XMLSchema> .
127 3 Mi Tian
  @prefix vamp:     <http://purl.org/ontology/vamp/> .
128 3 Mi Tian
  @prefix :         <#> .
129 3 Mi Tian
130 3 Mi Tian
 
131 3 Mi Tian
:transform_plugin a vamp:Plugin ;
132 3 Mi Tian
    vamp:identifier "vampy-onsets-cdsf" .
133 3 Mi Tian
134 3 Mi Tian
:transform_library a vamp:PluginLibrary ;
135 3 Mi Tian
    vamp:identifier "vampy" ;
136 3 Mi Tian
    vamp:available_plugin :transform_plugin .
137 3 Mi Tian
138 3 Mi Tian
:transform a vamp:Transform ;
139 3 Mi Tian
    vamp:plugin :transform_plugin ;
140 3 Mi Tian
    vamp:step_size "512"^^xsd:int ;
141 3 Mi Tian
    vamp:block_size "1024"^^xsd:int ;
142 3 Mi Tian
    vamp:parameter_binding [
143 3 Mi Tian
        vamp:parameter [ vamp:identifier "bt-threshold" ] ;
144 3 Mi Tian
        vamp:value "0.9"^^xsd:float ;
145 3 Mi Tian
    ] ;
146 3 Mi Tian
    vamp:parameter_binding [
147 3 Mi Tian
        vamp:parameter [ vamp:identifier "cut-off" ] ;
148 3 Mi Tian
        vamp:value "0.34"^^xsd:float ;
149 3 Mi Tian
    ] ;
150 3 Mi Tian
    vamp:parameter_binding [
151 3 Mi Tian
        vamp:parameter [ vamp:identifier "dthreshold" ] ;
152 3 Mi Tian
        vamp:value "0"^^xsd:float ;
153 3 Mi Tian
    ] ;
154 3 Mi Tian
    vamp:parameter_binding [
155 3 Mi Tian
        vamp:parameter [ vamp:identifier "filtfilt" ] ;
156 3 Mi Tian
        vamp:value "1"^^xsd:float ;
157 3 Mi Tian
    ] ;
158 3 Mi Tian
    vamp:parameter_binding [
159 3 Mi Tian
        vamp:parameter [ vamp:identifier "fusion-type" ] ;
160 3 Mi Tian
        vamp:value "2"^^xsd:float ;
161 3 Mi Tian
    ] ;
162 3 Mi Tian
    vamp:parameter_binding [
163 3 Mi Tian
        vamp:parameter [ vamp:identifier "lin-threshold" ] ;
164 3 Mi Tian
        vamp:value "0.5"^^xsd:float ;
165 3 Mi Tian
    ] ;
166 3 Mi Tian
    vamp:parameter_binding [
167 3 Mi Tian
        vamp:parameter [ vamp:identifier "med-threshold" ] ;
168 3 Mi Tian
        vamp:value "7"^^xsd:float ;
169 3 Mi Tian
    ] ;
170 3 Mi Tian
    vamp:parameter_binding [
171 3 Mi Tian
        vamp:parameter [ vamp:identifier "medfilt" ] ;
172 3 Mi Tian
        vamp:value "1"^^xsd:float ;
173 3 Mi Tian
    ] ;
174 3 Mi Tian
    vamp:parameter_binding [
175 3 Mi Tian
        vamp:parameter [ vamp:identifier "polyfit" ] ;
176 3 Mi Tian
        vamp:value "1"^^xsd:float ;
177 3 Mi Tian
    ] ;
178 3 Mi Tian
    vamp:parameter_binding [
179 3 Mi Tian
        vamp:parameter [ vamp:identifier "threshold" ] ;
180 3 Mi Tian
        vamp:value "50"^^xsd:float ;
181 3 Mi Tian
    ] ;
182 3 Mi Tian
    vamp:parameter_binding [
183 3 Mi Tian
        vamp:parameter [ vamp:identifier "tol-threshold" ] ;
184 1 Mi Tian
        vamp:value "0.03"^^xsd:float ;
185 1 Mi Tian
    ] ;
186 1 Mi Tian
    vamp:parameter_binding [
187 1 Mi Tian
        vamp:parameter [ vamp:identifier "whitening" ] ;
188 1 Mi Tian
        vamp:value "0"^^xsd:float ;
189 1 Mi Tian
    ] ;
190 1 Mi Tian
    vamp:output [ vamp:identifier "vampy-onsets-cdsf-onsets" ] .
191 1 Mi Tian
mitian$
192 1 Mi Tian
</pre>
193 1 Mi Tian
194 1 Mi Tian
Transforms are usually described in _RDF_, following the transform part of the _Vamp plugin ontology_. Now, how can be change the parameter settings in the transform specification file? 
195 1 Mi Tian
196 4 Mi Tian
The solution is to save this RDF file and modify any content in it as we wish, then to run this transform by specifying the saved file with -t options. Let's see an example.
197 1 Mi Tian
198 4 Mi Tian
<pre>
199 4 Mi Tian
mitian$
200 4 Mi Tian
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets >param_config.n3
201 4 Mi Tian
mitian$
202 4 Mi Tian
</pre>
203 4 Mi Tian
Now the transform file has been saved to the .n3 (RDF) file. Say, we want to use a higher threshold and change it from 50 to 70. Open an text editor, simple modify the corresponding line
204 4 Mi Tian
<pre>
205 4 Mi Tian
        vamp:parameter [ vamp:identifier "threshold" ] ;
206 4 Mi Tian
        vamp:value "50"^^xsd:float ;
207 4 Mi Tian
</pre>
208 4 Mi Tian
into 
209 4 Mi Tian
<pre>
210 4 Mi Tian
        vamp:parameter [ vamp:identifier "threshold" ] ;
211 4 Mi Tian
        vamp:value "70"^^xsd:float ;
212 4 Mi Tian
</pre>
213 4 Mi Tian
and save it.
214 4 Mi Tian
215 4 Mi Tian
Now we can run the same plugin using the modified parameter settings.
216 4 Mi Tian
<pre>
217 4 Mi Tian
mitian$
218 4 Mi Tian
mitian$ sonic-annotator -t param_config.n3 -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
219 4 Mi Tian
Extracting and writing features... Done
220 4 Mi Tian
221 4 Mi Tian
Vampy::~PyExtensionManager: Extension module cleaned.
222 4 Mi Tian
mitian$
223 4 Mi Tian
mitian$
224 4 Mi Tian
</pre>
225 4 Mi Tian
226 4 Mi Tian
h2. That's all guys! Let's hack with Vamp plugins :)