Wiki » History » Version 6

Mi Tian, 2015-08-05 09:34 PM

1 1 Mi Tian
h1. Wiki
2 1 Mi Tian
3 1 Mi Tian
h2. Vamp environment for feature extraction
4 1 Mi Tian
5 4 Mi Tian
_Vamp_ is an audio processing plugin system for plugins that extract descriptive information from audio data — typically referred to as audio analysis plugins or audio feature extraction plugins. The _vamp plugins_ are distributed in shared library files with extension _.dll_, _.so_, or _.dylib_ depending on the platform.  
6 1 Mi Tian
7 4 Mi Tian
Several common audio file formats are supported, including MP3, Ogg, and a number of PCM formats such as WAV and AIFF. AAC is supported on OS/X only, and only if not DRM protected. WMA is not supported.
8 1 Mi Tian
9 5 Mi Tian
See [[http://vamp-plugins.org/]] for more introduction about Vamp environment and the plugins.
10 5 Mi Tian
11 1 Mi Tian
h2. Host applications
12 1 Mi Tian
13 4 Mi Tian
A vamp plugin cannot be used on its own, but only with a conforming _host_ application which loads the plugin from its shared library and calls functions within the plugin code to configure it, supply it with data, and run it.
14 4 Mi Tian
15 6 Mi Tian
* [["Sonic Visualiser":http://www.sonicvisualiser.org/]] -- An interactive audio analysis and visualisation tool using Vamp plugins
16 1 Mi Tian
17 6 Mi Tian
* [["Sonic Annotator":https://code.soundsoftware.ac.uk/projects/sonic-annotator]] -- A command line tool facilitating facilitating feature extraction, parameter configuration and output manipulation for a large bunch of audio files.
18 6 Mi Tian
19 6 Mi Tian
* [["Audacity":http://audacityteam.org/]] -- An audio editor which enables recording, labelling and editing the audio.
20 6 Mi Tian
21 6 Mi Tian
22 4 Mi Tian
h2. Plugin installation
23 1 Mi Tian
24 1 Mi Tian
The vamp plugins need to be put in a directory where the host can locate, i.e., the _VAMP_PATH_. See [[http://vamp-plugins.org/download.html#install]] for installation instructions. 
25 1 Mi Tian
26 5 Mi Tian
Alternatively, you can set your own VAMP_PATH environment variable to list the locations a host should look in for Vamp plugins. For example, the following command will set @/homes/mitian/hg/onset-fusion@ as an alternative VAMP_PATH. Plugins placed in this directory will also be locatable for the hosts besides those under the system specified vamp directories.
27 4 Mi Tian
<pre>
28 5 Mi Tian
mitian$ export VAMP_PATH=$VAMP_PATH:/homes/mitian/hg/onset-fusion
29 4 Mi Tian
</pre>
30 4 Mi Tian
31 4 Mi Tian
Once the plugins are installed in the Vamp path, they will show themselves under the _Transform_ tab in Sonic Visualiser, or the _Analyze_ tab in Audacity, or in the terminal when you list them using Sonic Annotator (@ sonic-annotator -l@). 
32 4 Mi Tian
33 1 Mi Tian
h2. Parameters and outputs of this plugin set
34 4 Mi Tian
35 4 Mi Tian
h3. Parameters
36 4 Mi Tian
<pre>
37 5 Mi Tian
    **Adjustable parameters**
38 4 Mi Tian
	Fusion Type 					   Fusion type [0: early fusion, 1: linear fusion, 2: decision fusion]
39 4 Mi Tian
	Detection Sensitivity   	    Sensitivity level for peak detection in the onset likelihood function.
40 4 Mi Tian
								       The higher the sensitivity, the more onsets will (rightly or wrongly) be detected.
41 4 Mi Tian
	Delta Threshold  			   Constant offset used for adaptive theresholding using the median of the detection function.
42 4 Mi Tian
	Backtracking Threshold  		   Backtracking threshold used determine the stopping condition for backtracking the detected onset location
43 4 Mi Tian
									to an earlier 'perceived' location.
44 1 Mi Tian
	Tolerance window				   Tolerance window to combine events from different detection functions for decision fusion.
45 4 Mi Tian
	Linear combination coefficient	 Combination coefficient for linear fusion.
46 4 Mi Tian
	Median filter window			   Median filter window size.
47 4 Mi Tian
	Cutoff frequency				   Cutoff frequency for the low pass filter.
48 4 Mi Tian
	
49 5 Mi Tian
	**bool parameters**		      [0: off, 1: on]
50 4 Mi Tian
	Polynomial fitting				 Use polynomial fitting to evaluate detection function peaks.
51 4 Mi Tian
	Median filtering				   Use median filter based adaptive thresholding.
52 4 Mi Tian
	Low-pass filtering				 Use low pass filtering to smooth onset detection functions.
53 4 Mi Tian
	Adaptive whitening				 Use adaptive whitening. It evens out the temporal and frequency variation in the signal.
54 4 Mi Tian
55 4 Mi Tian
</pre>
56 4 Mi Tian
57 4 Mi Tian
h3. Outputs
58 4 Mi Tian
59 1 Mi Tian
60 6 Mi Tian
# *Note Onsets* – The detected note onset times, returned as a single feature with timestamp.
61 6 Mi Tian
# *Smoothed Detection Function* – The smoothed onset detection function.
62 6 Mi Tian
# *Note Onsets* – The detected note onset times, returned as a single feature with timestamp.
63 1 Mi Tian
64 1 Mi Tian
65 5 Mi Tian
NOTE: The last two outputs are not available in fusion based onset detectors. Please check individual plugins for specific output types.
66 1 Mi Tian
 
67 1 Mi Tian
h2. Example usages of the onset detectors
68 1 Mi Tian
69 5 Mi Tian
To use these onset detector plugins in Sonic Visualiser and Audacity is quite straightforward. Load your audio file first, then select a feature extractor/plugin from the transform list. You will also find a set of adjustable parameters in the pop-out box associated with each transform. The extracted features, whether it is onset locations or detection functions, will be visualised in the pane on top of the waveform, synchronised with time.
70 5 Mi Tian
 
71 5 Mi Tian
For comprehensive user manuals, see
72 5 Mi Tian
[[http://www.sonicvisualiser.org/doc/reference/2.1/en/index.html]]  
73 5 Mi Tian
[[http://wiki.audacityteam.org/wiki/Category:Tutorial]]
74 5 Mi Tian
75 6 Mi Tian
Rather than for visualisation purpose, Sonic Annotator lets you to extract and store audio features efficiently. Here we give a quick tutorial of how to detect onset for a given audio file and save the results to local folder using Sonic Annotator.
76 3 Mi Tian
<pre>
77 3 Mi Tian
mitian$ sonic-annotator -l | grep vampy-onsets
78 3 Mi Tian
vamp:vampy:vampy-onsets-bersf:vampy-onsets-bersf-onsets
79 3 Mi Tian
vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
80 3 Mi Tian
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-df
81 3 Mi Tian
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-onsets
82 3 Mi Tian
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-sdf
83 3 Mi Tian
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-df
84 3 Mi Tian
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-onsets
85 3 Mi Tian
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-sdf
86 3 Mi Tian
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-df
87 3 Mi Tian
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-onsets
88 3 Mi Tian
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-sdf
89 3 Mi Tian
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-df
90 3 Mi Tian
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-onsets
91 3 Mi Tian
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-sdf
92 1 Mi Tian
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-df
93 3 Mi Tian
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-onsets
94 3 Mi Tian
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-sdf
95 3 Mi Tian
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-df
96 3 Mi Tian
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-onsets
97 3 Mi Tian
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-sdf
98 1 Mi Tian
mitian$
99 1 Mi Tian
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
100 3 Mi Tian
Extracting and writing features... Done
101 4 Mi Tian
102 3 Mi Tian
Vampy::~PyExtensionManager: Extension module cleaned.
103 3 Mi Tian
mitian$
104 1 Mi Tian
mitian$
105 3 Mi Tian
</pre>
106 3 Mi Tian
107 6 Mi Tian
Each line of the output csv file consists of the timestamp (and duration where applicable) for a single feature, the value of that feature and the feature's label (where applicable), separated by comma.
108 6 Mi Tian
109 3 Mi Tian
Sonic Anntator also allows for batch processing in one go. 
110 3 Mi Tian
<pre>
111 1 Mi Tian
mitian$
112 4 Mi Tian
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv /homes/mitian/documents/audio/*.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
113 1 Mi Tian
Extracting and writing features... Done
114 3 Mi Tian
115 3 Mi Tian
Vampy::~PyExtensionManager: Extension module cleaned.
116 3 Mi Tian
mitian$
117 3 Mi Tian
mitian$
118 3 Mi Tian
</pre>
119 3 Mi Tian
120 4 Mi Tian
h2. Play around with your vamp plugin feature extractors
121 3 Mi Tian
122 4 Mi Tian
We may want use parameter settings other the default. With Sonic Visualiser or Audacity, you can easily access the GUI to modify them. Whiles with Sonic Annotator, you would need to interact with 
123 3 Mi Tian
124 4 Mi Tian
Firstly, we can check the current parameter configurations in the skeleton transform description using Sonic Annotator's @-s@ command.
125 3 Mi Tian
126 3 Mi Tian
<pre>
127 3 Mi Tian
mitian$
128 3 Mi Tian
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
129 3 Mi Tian
@prefix xsd:      <http://www.w3.org/2001/XMLSchema> .
130 3 Mi Tian
  @prefix vamp:     <http://purl.org/ontology/vamp/> .
131 3 Mi Tian
  @prefix :         <#> .
132 3 Mi Tian
133 3 Mi Tian
 
134 3 Mi Tian
:transform_plugin a vamp:Plugin ;
135 3 Mi Tian
    vamp:identifier "vampy-onsets-cdsf" .
136 3 Mi Tian
137 3 Mi Tian
:transform_library a vamp:PluginLibrary ;
138 3 Mi Tian
    vamp:identifier "vampy" ;
139 3 Mi Tian
    vamp:available_plugin :transform_plugin .
140 3 Mi Tian
141 3 Mi Tian
:transform a vamp:Transform ;
142 3 Mi Tian
    vamp:plugin :transform_plugin ;
143 3 Mi Tian
    vamp:step_size "512"^^xsd:int ;
144 3 Mi Tian
    vamp:block_size "1024"^^xsd:int ;
145 3 Mi Tian
    vamp:parameter_binding [
146 3 Mi Tian
        vamp:parameter [ vamp:identifier "bt-threshold" ] ;
147 3 Mi Tian
        vamp:value "0.9"^^xsd:float ;
148 3 Mi Tian
    ] ;
149 3 Mi Tian
    vamp:parameter_binding [
150 3 Mi Tian
        vamp:parameter [ vamp:identifier "cut-off" ] ;
151 3 Mi Tian
        vamp:value "0.34"^^xsd:float ;
152 3 Mi Tian
    ] ;
153 3 Mi Tian
    vamp:parameter_binding [
154 3 Mi Tian
        vamp:parameter [ vamp:identifier "dthreshold" ] ;
155 3 Mi Tian
        vamp:value "0"^^xsd:float ;
156 3 Mi Tian
    ] ;
157 3 Mi Tian
    vamp:parameter_binding [
158 3 Mi Tian
        vamp:parameter [ vamp:identifier "filtfilt" ] ;
159 3 Mi Tian
        vamp:value "1"^^xsd:float ;
160 3 Mi Tian
    ] ;
161 3 Mi Tian
    vamp:parameter_binding [
162 3 Mi Tian
        vamp:parameter [ vamp:identifier "fusion-type" ] ;
163 3 Mi Tian
        vamp:value "2"^^xsd:float ;
164 3 Mi Tian
    ] ;
165 3 Mi Tian
    vamp:parameter_binding [
166 3 Mi Tian
        vamp:parameter [ vamp:identifier "lin-threshold" ] ;
167 3 Mi Tian
        vamp:value "0.5"^^xsd:float ;
168 3 Mi Tian
    ] ;
169 3 Mi Tian
    vamp:parameter_binding [
170 3 Mi Tian
        vamp:parameter [ vamp:identifier "med-threshold" ] ;
171 3 Mi Tian
        vamp:value "7"^^xsd:float ;
172 3 Mi Tian
    ] ;
173 3 Mi Tian
    vamp:parameter_binding [
174 3 Mi Tian
        vamp:parameter [ vamp:identifier "medfilt" ] ;
175 3 Mi Tian
        vamp:value "1"^^xsd:float ;
176 3 Mi Tian
    ] ;
177 3 Mi Tian
    vamp:parameter_binding [
178 3 Mi Tian
        vamp:parameter [ vamp:identifier "polyfit" ] ;
179 3 Mi Tian
        vamp:value "1"^^xsd:float ;
180 3 Mi Tian
    ] ;
181 3 Mi Tian
    vamp:parameter_binding [
182 3 Mi Tian
        vamp:parameter [ vamp:identifier "threshold" ] ;
183 3 Mi Tian
        vamp:value "50"^^xsd:float ;
184 3 Mi Tian
    ] ;
185 3 Mi Tian
    vamp:parameter_binding [
186 1 Mi Tian
        vamp:parameter [ vamp:identifier "tol-threshold" ] ;
187 1 Mi Tian
        vamp:value "0.03"^^xsd:float ;
188 1 Mi Tian
    ] ;
189 1 Mi Tian
    vamp:parameter_binding [
190 1 Mi Tian
        vamp:parameter [ vamp:identifier "whitening" ] ;
191 1 Mi Tian
        vamp:value "0"^^xsd:float ;
192 1 Mi Tian
    ] ;
193 1 Mi Tian
    vamp:output [ vamp:identifier "vampy-onsets-cdsf-onsets" ] .
194 1 Mi Tian
mitian$
195 1 Mi Tian
</pre>
196 1 Mi Tian
197 6 Mi Tian
Transforms are usually described in _RDF_, following the transform part of the [["Vamp plugin ontology":http://www.omras2.org/VampOntology]]. Now, how can be change the parameter settings in the transform specification file? 
198 1 Mi Tian
199 6 Mi Tian
The solution is to save this RDF file and modify any content in it as we wish, then to run this transform by specifying the saved file with the @-t@ option. Let's see an example.
200 1 Mi Tian
201 4 Mi Tian
<pre>
202 4 Mi Tian
mitian$
203 4 Mi Tian
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets >param_config.n3
204 4 Mi Tian
mitian$
205 4 Mi Tian
</pre>
206 6 Mi Tian
Now the transform file has been saved to the .n3 (RDF) file. Say, we want to use a higher threshold and change it from 50 to 70. Open an text editor, simple modify the corresponding lines in the saved n3 file
207 4 Mi Tian
<pre>
208 4 Mi Tian
        vamp:parameter [ vamp:identifier "threshold" ] ;
209 4 Mi Tian
        vamp:value "50"^^xsd:float ;
210 4 Mi Tian
</pre>
211 4 Mi Tian
into 
212 4 Mi Tian
<pre>
213 4 Mi Tian
        vamp:parameter [ vamp:identifier "threshold" ] ;
214 4 Mi Tian
        vamp:value "70"^^xsd:float ;
215 4 Mi Tian
</pre>
216 1 Mi Tian
and save it.
217 1 Mi Tian
218 1 Mi Tian
Now we can run the same plugin using the modified parameter settings.
219 1 Mi Tian
<pre>
220 1 Mi Tian
mitian$
221 4 Mi Tian
mitian$ sonic-annotator -t param_config.n3 -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
222 4 Mi Tian
Extracting and writing features... Done
223 4 Mi Tian
224 4 Mi Tian
Vampy::~PyExtensionManager: Extension module cleaned.
225 4 Mi Tian
mitian$
226 4 Mi Tian
mitian$
227 4 Mi Tian
</pre>
228 4 Mi Tian
229 6 Mi Tian
This example only involves a couple of operations for Sonic Annotator. You may also want to type
230 6 Mi Tian
@sonic-annotator -h@
231 6 Mi Tian
for a helpbook.
232 6 Mi Tian
  
233 6 Mi Tian
h2. That's all guys! Happy hacking with Vamp plugins :)