Wiki » History » Version 4
Version 3 (Mi Tian, 2015-08-05 05:51 PM) → Version 4/18 (Mi Tian, 2015-08-05 08:36 PM)
h1. Wiki
h2. Vamp environment for feature extraction
_Vamp_ is an audio processing plugin system for plugins that extract descriptive information from audio data — typically referred to as audio analysis plugins or audio feature extraction plugins. The _vamp plugins_ are distributed in shared library files with extension _.dll_, _.so_, or _.dylib_ depending on the platform.
See [[http://vamp-plugins.org/]] for more details about Vamp plugins.
Several common audio file formats are supported, including MP3, Ogg, and a number of PCM formats such as WAV and AIFF. AAC is supported on OS/X only, and only if not DRM protected. WMA is not supported.
h2. Host applications
A vamp plugin cannot be used on its own, but only with a conforming _host_ application which loads the plugin from its shared library and calls functions within the plugin code to configure it, supply it with data, and run it.
See [[http://vamp-plugins.org/]] for more details about Vamp plugins.
h2. Host applications
* Sonic Visualiser -- An interactive audio analysis and visualisation tool using Vamp plugins
[[http://www.sonicvisualiser.org/]]
* Sonic Annotator -- A command line tool facilitating facilitating feature extraction, parameter configuration and output manipulation for a large bunch of audio files.
[[https://code.soundsoftware.ac.uk/projects/sonic-annotator]]
* Audacity -- An audio editor which enables recording, labelling and editing the audio.
[[http://audacityteam.org/]]
h2. Plugin installation
The vamp plugins need to be put in a directory where the host can locate, i.e., the _VAMP_PATH_. See [[http://vamp-plugins.org/download.html#install]] for installation instructions.
Alternatively, you can set your own VAMP_PATH environment variable to list the locations a host should look in for Vamp plugins. For example, the following command will set @/homes/mtian/hg/onset-fusion@ as an alternative VAMP_PATH. Plugins placed in this directory will also be locatable for the hosts besides those under the system specified vamp directories.
<pre>
mitian$ export VAMP_PATH=$VAMP_PATH:/homes/mtian/hg/onset-fusion
</pre>
Once the plugins are installed in the Vamp path, they will show themselves under the _Transform_ tab in Sonic Visualiser, or the _Analyze_ tab in Audacity, or in the terminal when you list them using Sonic Annotator (@ sonic-annotator -l@). and
h2. Parameters and outputs of this plugin set
h3. Parameters
<pre>
Fusion Type Fusion type [0: early fusion, 1: linear fusion, 2: decision fusion]
Detection Sensitivity Sensitivity level for peak detection in the onset likelihood function.
The higher the sensitivity, the more onsets will (rightly or wrongly) be detected.
Delta Threshold Constant offset used for adaptive theresholding using the median of the detection function.
Backtracking Threshold Backtracking threshold used determine the stopping condition for backtracking the detected onset location
to an earlier 'perceived' location.
Tolerance window Tolerance window to combine events from different detection functions for decision fusion.
Linear combination coefficient Combination coefficient for linear fusion.
Median filter window Median filter window size.
Cutoff frequency Cutoff frequency for the low pass filter.
**bool parameters** [0: off, 1: on]
Polynomial fitting Use polynomial fitting to evaluate detection function peaks.
Median filtering Use median filter based adaptive thresholding.
Low-pass filtering Use low pass filtering to smooth onset detection functions.
Adaptive whitening Use adaptive whitening. It evens out the temporal and frequency variation in the signal.
</pre>
h3. Outputs
# Note Onsets – The detected note onset times, returned as a single feature with timestamp.
# Smoothed Detection Function – The smoothed onset detection function.
# Note Onsets – The detected note onset times, returned as a single feature with timestamp.
NOTE: The last two outputs are not available in fusion based onset detectors.
h2. Example usages of the onset detectors
<pre>
mitian$ sonic-annotator -l | grep vampy-onsets
vamp:vampy:vampy-onsets-bersf:vampy-onsets-bersf-onsets
vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-df
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-onsets
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-sdf
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-df
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-onsets
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-sdf
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-df
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-onsets
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-sdf
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-df
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-onsets
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-sdf
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-df
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-onsets
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-sdf
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-df
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-onsets
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-sdf
mitian$
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection /homes/mt302/hg/workspace/onsetdetection
Extracting and writing features... Done
Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>
Sonic Anntator also allows for batch processing in one go.
<pre>
mitian$
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv /homes/mitian/documents/audio/*.wav /homes/mt302/documents/audio/*.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection /homes/mt302/hg/workspace/onsetdetection
Extracting and writing features... Done
Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>
h2. Play around with your vamp plugin feature extractors with _Vamp Ontologies_
We may want use parameter settings other the default. With Sonic Visualiser or Audacity, you users can easily access the GUI to modify them. Whiles with Sonic Annotator, you would need to interact with
Firstly, we can check the current parameter configurations in the skeleton transform description using Sonic Annotator's @-s@ command. cammand.
<pre>
mitian$
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
@prefix xsd: <http://www.w3.org/2001/XMLSchema> .
@prefix vamp: <http://purl.org/ontology/vamp/> .
@prefix : <#> .
:transform_plugin a vamp:Plugin ;
vamp:identifier "vampy-onsets-cdsf" .
:transform_library a vamp:PluginLibrary ;
vamp:identifier "vampy" ;
vamp:available_plugin :transform_plugin .
:transform a vamp:Transform ;
vamp:plugin :transform_plugin ;
vamp:step_size "512"^^xsd:int ;
vamp:block_size "1024"^^xsd:int ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "bt-threshold" ] ;
vamp:value "0.9"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "cut-off" ] ;
vamp:value "0.34"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "dthreshold" ] ;
vamp:value "0"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "filtfilt" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "fusion-type" ] ;
vamp:value "2"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "lin-threshold" ] ;
vamp:value "0.5"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "med-threshold" ] ;
vamp:value "7"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "medfilt" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "polyfit" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "threshold" ] ;
vamp:value "50"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "tol-threshold" ] ;
vamp:value "0.03"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "whitening" ] ;
vamp:value "0"^^xsd:float ;
] ;
vamp:output [ vamp:identifier "vampy-onsets-cdsf-onsets" ] .
mitian$
</pre>
Transforms are usually described in _RDF_, following the transform part of the _Vamp plugin ontology_. Now, how can be change the parameter settings in the transform specification file?
The solution is to save this RDF file and modify any content in it as we wish, then to run this transform by specifying the saved file with -t options. Let's see an example.
<pre>
mitian$
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets >param_config.n3
mitian$
</pre>
Now the transform file has been saved to the .n3 (RDF) file. Say, we want to use a higher threshold and change it from 50 to 70. Open an text editor, simple modify the corresponding line
<pre>
vamp:parameter [ vamp:identifier "threshold" ] ;
vamp:value "50"^^xsd:float ;
</pre>
into
<pre>
vamp:parameter [ vamp:identifier "threshold" ] ;
vamp:value "70"^^xsd:float ;
</pre>
and save it.
Now we can run the same plugin using the modified parameter settings.
<pre>
mitian$
mitian$ sonic-annotator -t param_config.n3 -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
Extracting and writing features... Done
Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>
h2. That's
h2.That's all guys! Let's hack with Vamp plugins :)
h2. Vamp environment for feature extraction
_Vamp_ is an audio processing plugin system for plugins that extract descriptive information from audio data — typically referred to as audio analysis plugins or audio feature extraction plugins. The _vamp plugins_ are distributed in shared library files with extension _.dll_, _.so_, or _.dylib_ depending on the platform.
See [[http://vamp-plugins.org/]] for more details about Vamp plugins.
Several common audio file formats are supported, including MP3, Ogg, and a number of PCM formats such as WAV and AIFF. AAC is supported on OS/X only, and only if not DRM protected. WMA is not supported.
h2. Host applications
A vamp plugin cannot be used on its own, but only with a conforming _host_ application which loads the plugin from its shared library and calls functions within the plugin code to configure it, supply it with data, and run it.
See [[http://vamp-plugins.org/]] for more details about Vamp plugins.
h2. Host applications
* Sonic Visualiser -- An interactive audio analysis and visualisation tool using Vamp plugins
[[http://www.sonicvisualiser.org/]]
* Sonic Annotator -- A command line tool facilitating facilitating feature extraction, parameter configuration and output manipulation for a large bunch of audio files.
[[https://code.soundsoftware.ac.uk/projects/sonic-annotator]]
* Audacity -- An audio editor which enables recording, labelling and editing the audio.
[[http://audacityteam.org/]]
h2. Plugin installation
The vamp plugins need to be put in a directory where the host can locate, i.e., the _VAMP_PATH_. See [[http://vamp-plugins.org/download.html#install]] for installation instructions.
Alternatively, you can set your own VAMP_PATH environment variable to list the locations a host should look in for Vamp plugins. For example, the following command will set @/homes/mtian/hg/onset-fusion@ as an alternative VAMP_PATH. Plugins placed in this directory will also be locatable for the hosts besides those under the system specified vamp directories.
<pre>
mitian$ export VAMP_PATH=$VAMP_PATH:/homes/mtian/hg/onset-fusion
</pre>
Once the plugins are installed in the Vamp path, they will show themselves under the _Transform_ tab in Sonic Visualiser, or the _Analyze_ tab in Audacity, or in the terminal when you list them using Sonic Annotator (@ sonic-annotator -l@). and
h2. Parameters and outputs of this plugin set
h3. Parameters
<pre>
Fusion Type Fusion type [0: early fusion, 1: linear fusion, 2: decision fusion]
Detection Sensitivity Sensitivity level for peak detection in the onset likelihood function.
The higher the sensitivity, the more onsets will (rightly or wrongly) be detected.
Delta Threshold Constant offset used for adaptive theresholding using the median of the detection function.
Backtracking Threshold Backtracking threshold used determine the stopping condition for backtracking the detected onset location
to an earlier 'perceived' location.
Tolerance window Tolerance window to combine events from different detection functions for decision fusion.
Linear combination coefficient Combination coefficient for linear fusion.
Median filter window Median filter window size.
Cutoff frequency Cutoff frequency for the low pass filter.
**bool parameters** [0: off, 1: on]
Polynomial fitting Use polynomial fitting to evaluate detection function peaks.
Median filtering Use median filter based adaptive thresholding.
Low-pass filtering Use low pass filtering to smooth onset detection functions.
Adaptive whitening Use adaptive whitening. It evens out the temporal and frequency variation in the signal.
</pre>
h3. Outputs
# Note Onsets – The detected note onset times, returned as a single feature with timestamp.
# Smoothed Detection Function – The smoothed onset detection function.
# Note Onsets – The detected note onset times, returned as a single feature with timestamp.
NOTE: The last two outputs are not available in fusion based onset detectors.
h2. Example usages of the onset detectors
<pre>
mitian$ sonic-annotator -l | grep vampy-onsets
vamp:vampy:vampy-onsets-bersf:vampy-onsets-bersf-onsets
vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-df
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-onsets
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-sdf
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-df
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-onsets
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-sdf
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-df
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-onsets
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-sdf
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-df
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-onsets
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-sdf
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-df
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-onsets
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-sdf
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-df
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-onsets
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-sdf
mitian$
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection /homes/mt302/hg/workspace/onsetdetection
Extracting and writing features... Done
Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>
Sonic Anntator also allows for batch processing in one go.
<pre>
mitian$
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv /homes/mitian/documents/audio/*.wav /homes/mt302/documents/audio/*.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection /homes/mt302/hg/workspace/onsetdetection
Extracting and writing features... Done
Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>
h2. Play around with your vamp plugin feature extractors with _Vamp Ontologies_
We may want use parameter settings other the default. With Sonic Visualiser or Audacity, you users can easily access the GUI to modify them. Whiles with Sonic Annotator, you would need to interact with
Firstly, we can check the current parameter configurations in the skeleton transform description using Sonic Annotator's @-s@ command. cammand.
<pre>
mitian$
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
@prefix xsd: <http://www.w3.org/2001/XMLSchema> .
@prefix vamp: <http://purl.org/ontology/vamp/> .
@prefix : <#> .
:transform_plugin a vamp:Plugin ;
vamp:identifier "vampy-onsets-cdsf" .
:transform_library a vamp:PluginLibrary ;
vamp:identifier "vampy" ;
vamp:available_plugin :transform_plugin .
:transform a vamp:Transform ;
vamp:plugin :transform_plugin ;
vamp:step_size "512"^^xsd:int ;
vamp:block_size "1024"^^xsd:int ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "bt-threshold" ] ;
vamp:value "0.9"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "cut-off" ] ;
vamp:value "0.34"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "dthreshold" ] ;
vamp:value "0"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "filtfilt" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "fusion-type" ] ;
vamp:value "2"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "lin-threshold" ] ;
vamp:value "0.5"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "med-threshold" ] ;
vamp:value "7"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "medfilt" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "polyfit" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "threshold" ] ;
vamp:value "50"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "tol-threshold" ] ;
vamp:value "0.03"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "whitening" ] ;
vamp:value "0"^^xsd:float ;
] ;
vamp:output [ vamp:identifier "vampy-onsets-cdsf-onsets" ] .
mitian$
</pre>
Transforms are usually described in _RDF_, following the transform part of the _Vamp plugin ontology_. Now, how can be change the parameter settings in the transform specification file?
The solution is to save this RDF file and modify any content in it as we wish, then to run this transform by specifying the saved file with -t options. Let's see an example.
<pre>
mitian$
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets >param_config.n3
mitian$
</pre>
Now the transform file has been saved to the .n3 (RDF) file. Say, we want to use a higher threshold and change it from 50 to 70. Open an text editor, simple modify the corresponding line
<pre>
vamp:parameter [ vamp:identifier "threshold" ] ;
vamp:value "50"^^xsd:float ;
</pre>
into
<pre>
vamp:parameter [ vamp:identifier "threshold" ] ;
vamp:value "70"^^xsd:float ;
</pre>
and save it.
Now we can run the same plugin using the modified parameter settings.
<pre>
mitian$
mitian$ sonic-annotator -t param_config.n3 -w csv test.wav --csv-basedir /homes/mitian/hg/workspace/onsetdetection
Extracting and writing features... Done
Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>
h2. That's
h2.That's all guys! Let's hack with Vamp plugins :)