Audio Onset Detection based on Fusion

h1. Wiki

h2. Vamp environment for feature extraction

_Vamp_ is an audio processing plugin system for plugins that extract descriptive information from audio data — typically referred to as audio analysis plugins or audio feature extraction plugins. The _vamp plugins_ are distributed in shared library files with extension _.dll_, _.so_, or _.dylib_ depending on the platform.  A vamp plugin cannot be used on its own, but only with a conforming _host_ application which loads the plugin from its shared library and calls functions within the plugin code to configure it, supply it with data, and run it.

See [[http://vamp-plugins.org/]] for more details about Vamp plugins.

h2. Host applications

* Sonic Visualiser

[[http://www.sonicvisualiser.org/]]

* Sonic Annotator 

[[https://code.soundsoftware.ac.uk/projects/sonic-annotator]]

* Audacity

[[http://audacityteam.org/]]

Once the plugins and 

h2. Example usages of the onset detectors

<pre>

mitian$ sonic-annotator -l | grep vampy-onsets

vamp:vampy:vampy-onsets-bersf:vampy-onsets-bersf-onsets

vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets

vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-df

vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-onsets

vamp:vampy:vampy-onsets-ber:vampy-onsets-ber-sdf

vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-df

vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-onsets

vamp:vampy:vampy-onsets-cd:vampy-onsets-cd-sdf

vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-df

vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-onsets

vamp:vampy:vampy-onsets-hfc:vampy-onsets-hfc-sdf

vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-df

vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-onsets

vamp:vampy:vampy-onsets-pd:vampy-onsets-pd-sdf

vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-df

vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-onsets

vamp:vampy:vampy-onsets-sd:vampy-onsets-sd-sdf

vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-df

vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-onsets

vamp:vampy:vampy-onsets-sf:vampy-onsets-sf-sdf

mitian$

mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv test.wav --csv-basedir /homes/mt302/hg/workspace/onsetdetection

Extracting and writing features... Done

Vampy::~PyExtensionManager: Extension module cleaned.

mitian$

mitian$

</pre>

Sonic Anntator also allows for batch processing in one go. 

<pre>

mitian$

mitian$ sonic-annotator -d vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets -w csv /homes/mt302/documents/audio/*.wav --csv-basedir /homes/mt302/hg/workspace/onsetdetection

Extracting and writing features... Done

Vampy::~PyExtensionManager: Extension module cleaned.

mitian$

mitian$

</pre>

h2. Play around with your vamp plugin feature extractors with _Vamp Ontologies_

We may want use parameter settings other the default. With Sonic Visualiser or Audacity, users can easily access the GUI to modify them. Whiles  with Sonic Annotator, you would need to interact with 

Firstly, we can check the current parameter configurations using Sonic Annotator's @-s@ cammand.

<pre>

mitian$

mitian$ sonic-annotator -s vamp:vampy:vampy-onsets-cdsf:vampy-onsets-cdsf-onsets

@prefix xsd:      <http://www.w3.org/2001/XMLSchema> .

  @prefix vamp:     <http://purl.org/ontology/vamp/> .

  @prefix :         <#> .

:transform_plugin a vamp:Plugin ;

    vamp:identifier "vampy-onsets-cdsf" .

:transform_library a vamp:PluginLibrary ;

    vamp:identifier "vampy" ;

    vamp:available_plugin :transform_plugin .

:transform a vamp:Transform ;

    vamp:plugin :transform_plugin ;

    vamp:step_size "512"^^xsd:int ;

    vamp:block_size "1024"^^xsd:int ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "bt-threshold" ] ;

        vamp:value "0.9"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "cut-off" ] ;

        vamp:value "0.34"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "dthreshold" ] ;

        vamp:value "0"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "filtfilt" ] ;

        vamp:value "1"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "fusion-type" ] ;

        vamp:value "2"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "lin-threshold" ] ;

        vamp:value "0.5"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "med-threshold" ] ;

        vamp:value "7"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "medfilt" ] ;

        vamp:value "1"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "polyfit" ] ;

        vamp:value "1"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "threshold" ] ;

        vamp:value "50"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "tol-threshold" ] ;

        vamp:value "0.03"^^xsd:float ;

    ] ;

    vamp:parameter_binding [

        vamp:parameter [ vamp:identifier "whitening" ] ;

        vamp:value "0"^^xsd:float ;

    ] ;

    vamp:output [ vamp:identifier "vampy-onsets-cdsf-onsets" ] .

mitian$

</pre>

Transforms are usually described in _RDF_, following the transform part of the _Vamp plugin ontology_. Now, how can be change the parameter settings in the transform specification file? 

Say, we want to use a higher threshold and change it from 50 to 70. 

h2.That's all guys! :)