Audio Onset Detection based on Fusion

h1. Wiki

h2. Vamp environment for feature extraction

_Vamp_ is an audio processing plugin system for plugins that extract descriptive information from audio data — typically referred to as audio analysis plugins or audio feature extraction plugins. The _vamp plugins_ are distributed in shared library files with extension _.dll_, _.so_, or _.dylib_ depending on the platform. A vamp plugin cannot be used on its own, but only with a conforming _host_ application which loads the plugin from its shared library and calls functions within the plugin code to configure it, supply it with data, and run it.

See [[http://vamp-plugins.org/]] for more details about Vamp plugins.

h2. Host applications

* Sonic Visualiser
[[http://www.sonicvisualiser.org/]]
* Sonic Annotator
[[https://code.soundsoftware.ac.uk/projects/sonic-annotator]]
* Audacity
[[http://audacityteam.org/]]

Once the plugins and

h2. Example usages of the onset detectors

<pre>
mitian$ sonic-annotator -l | grep vampy-onsets
vamp:vampy:vampy-onsets-bersf:vampy-onsets-bersf-onsets
vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-df
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-onsets
vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-sdf
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-df
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-onsets
vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-sdf
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-df
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-onsets
vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-sdf
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-df
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-onsets
vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-sdf
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-df
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-onsets
vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-sdf
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-df
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-onsets
vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-sdf
mitian$
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv test.wav --csv-basedir /homes/mt302/hg/workspace/onsetdetection
Extracting and writing features... Done

Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>

Sonic Anntator You can also allows check [[https://code.soundsoftware.ac.uk/projects/sonic-annotator/wiki]] for batch processing in one go.
<pre>
mitian$
mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv /homes/mt302/documents/audio/*.wav --csv-basedir /homes/mt302/hg/workspace/onsetdetection
Extracting and writing features... Done a wiki for Sonic Annotator.

Vampy::~PyExtensionManager: Extension module cleaned.
mitian$
mitian$
</pre>

h2. Play around with your vamp plugin feature extractors with _Vamp Ontologies_

We may want use parameter settings other the default. With Sonic Visualiser or Audacity, users can easily access the GUI to modify them. Whiles with Sonic Annotator, you would need to interact with

Firstly, we can check the current parameter configurations using Sonic Annotator's @-s@ cammand.

<pre>
mitian$
mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets
@prefix xsd: <http://www.w3.org/2001/XMLSchema> .
@prefix vamp: <http://purl.org/ontology/vamp/> .
@prefix : <#> .


:transform_plugin a vamp:Plugin ;
vamp:identifier "vampy-onsets-cdsf" .

:transform_library a vamp:PluginLibrary ;
vamp:identifier "vampy" ;
vamp:available_plugin :transform_plugin .

:transform a vamp:Transform ;
vamp:plugin :transform_plugin ;
vamp:step_size "512"^^xsd:int ;
vamp:block_size "1024"^^xsd:int ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "bt-threshold" ] ;
vamp:value "0.9"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "cut-off" ] ;
vamp:value "0.34"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "dthreshold" ] ;
vamp:value "0"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "filtfilt" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "fusion-type" ] ;
vamp:value "2"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "lin-threshold" ] ;
vamp:value "0.5"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "med-threshold" ] ;
vamp:value "7"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "medfilt" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "polyfit" ] ;
vamp:value "1"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "threshold" ] ;
vamp:value "50"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "tol-threshold" ] ;
vamp:value "0.03"^^xsd:float ;
] ;
vamp:parameter_binding [
vamp:parameter [ vamp:identifier "whitening" ] ;
vamp:value "0"^^xsd:float ;
] ;
vamp:output [ vamp:identifier "vampy-onsets-cdsf-onsets" ] .
mitian$
</pre>

Transforms are usually described in _RDF_, following the transform part h2. Example usages of the _Vamp plugin ontology_. Now, how can be change the parameter settings in the transform specification file?

Say, we want to use a higher threshold and change it from 50 to 70.

h2.That's all guys! :) onset detectors