Audio Features and Vamp Plugins » History » Version 9

Chris Cannam, 2015-07-15 01:02 PM

1 1 Chris Cannam
h1. Audio Features and Vamp Plugins
2 1 Chris Cannam
3 1 Chris Cannam
h3. General outline
4 1 Chris Cannam
5 3 Chris Cannam
To be propagated to https://www.doc.gold.ac.uk/~mas01cr/teaching/dhoxss15/
6 3 Chris Cannam
7 1 Chris Cannam
# Introductory notes and slides on acoustics and audio (CR)
8 2 Chris Cannam
# Sonic Visualiser - hands on with waveform and spectrograms (CC)
9 1 Chris Cannam
# Introductory notes and slides on audio features (CC)
10 1 Chris Cannam
# Sonic Visualiser - hands on with Vamp plugins (CC)
11 1 Chris Cannam
# Python/IPython intro (CR)
12 3 Chris Cannam
# Feature Extraction using Vamp Plugins in Python (IPython Notebook) (CC)
13 3 Chris Cannam
# Audio Indexing and Search in Python (IPython Notebook) (CR)
14 3 Chris Cannam
15 3 Chris Cannam
h3. Breakdown of CC sections
16 3 Chris Cannam
17 4 Chris Cannam
h5. Sonic Visualiser - hands on with waveform and spectrograms
18 3 Chris Cannam
19 7 Chris Cannam
* Waveform
20 5 Chris Cannam
## Start Sonic Visualiser and open "A Friendly Warning"
21 6 Chris Cannam
## Show dragging through the file using Navigate tool, and also using the overview at bottom
22 6 Chris Cannam
## Play from the start, just to get an idea what it sounds like
23 5 Chris Cannam
## Return to the start and zoom in (using the zoom wheel, but noting that the mouse wheel also works)
24 6 Chris Cannam
## Notice the different shapes in waveform resulting from different types of synthetic percussive sound (low-frequency kick drum / higher frequency cymbal-type sounds) - refer back to Christophe's notes about correspondence between e.g. signal voltage and speaker cone deflection
25 1 Chris Cannam
## Continue until the vocal starts, and observe that we can see very little that relates to e.g. sung pitch, although if we zoom in we can quite clearly see sibilance (these frequencies around 10kHz are pretty much the sweet spot for visibility in a 44.1kHz waveform)
26 6 Chris Cannam
## We will return to this track
27 6 Chris Cannam
28 8 Chris Cannam
* Spectrogram
29 1 Chris Cannam
## New session, open piano-scale.wav and play it
30 9 Chris Cannam
## Some information can sort-of be perceived and measured from the waveform here: we can see when the notes start, and can get simple fundamental frequency estimate - zoom in to the first note, switch to Select mode, drag out one cycle - it's about 170 samples, so 44100/170 = 259 Hz - the note is a middle C so true value should be nearer to 261, but this is a fair approximation. (But this is a very simple example!)
31 9 Chris Cannam
## Now open a plain spectrogram - Pane -> Add Spectrogram (or G key). Observe full range on frequency scale; x axis is time, this is a simple time-frequency breakdown.
32 8 Chris Cannam
## Notice that, for each note, we can see the fundamental frequency most strongly and then the harmonic sequence. The harmonics are spaced more widely for higher notes because they are multiples of the fundamental frequency, which is larger. The noise floor is visible because we're using a dB scale, can switch to Linear to isolate only the strong frequencies. There isn't really enough detail to measure much here. (NB the default colour scheme is unhelpful to colour blind users, so it might be worth changing to Sunset scheme.)
33 9 Chris Cannam
## Close that pane and open a "melodic-range spectrogram" - Pane -> Add Melodic Range Spectrogram (or M key). Observe the much more limited frequency range and the fact that this spectrogram uses both Linear colour and Sunset scheme by default.
34 9 Chris Cannam
## Although the higher harmonics quickly disappear off the top of the scale, we can clearly see that the spacing between harmonics is now the same for each note, and that the semitones are equally-spaced (note spacing corresponding to the major scale intervals).
35 5 Chris Cannam
36 4 Chris Cannam
h5. Introductory notes and slides on audio features
37 3 Chris Cannam
38 4 Chris Cannam
h5. Sonic Visualiser - hands on with Vamp plugins
39 3 Chris Cannam
40 3 Chris Cannam
h5. Feature Extraction using Vamp Plugins in Python
41 3 Chris Cannam
42 3 Chris Cannam
h3. Materials
43 3 Chris Cannam
44 3 Chris Cannam
* Audio features slides: "PDF":https://code.soundsoftware.ac.uk/projects/dhoxss15/repository/raw/vamp-sv.pdf, "PowerPoint":https://code.soundsoftware.ac.uk/projects/dhoxss15/repository/raw/vamp-sv.pptx
45 3 Chris Cannam
46 3 Chris Cannam
* "IPython Notebook for Vamp Plugins in Python":https://code.soundsoftware.ac.uk/projects/dhoxss15/repository/raw/Vamp.v3.ipynb