annotate gmm_baseline_experiments/external_libs/librosa/librosa-0.3.1/docs/tutorial.rst @ 5:b523456082ca tip

Update path to dataset and reflect modified chunk naming convention.

Tutorial
========

This section covers the fundamentals of developing with *librosa*, including
a package overview, basic and advanced usage, and integration with the *scikit-learn*
package.  We will assume basic familiarity with Python and NumPy/SciPy.


Overview
--------

The *librosa* package is structured as collection of submodules:

  - librosa

    - :ref:`librosa.beat <beat>`
        Functions for estimating tempo and detecting beat events.

    - :ref:`librosa.chord <chord>`
        This submodule contains a generic class which implements supervised training
        of Gaussian-emission Hidden Markov Models (HMM) commonly used in chord
        recognition. 

    - :ref:`librosa.core <core>`
        Core functionality includes functions to load audio from disk, compute various
        spectrogram representations, and a variety of commonly used tools for
        music analysis.  For convenience, all functionality in this submodule is
        directly accessible from the top-level ``librosa.*`` namespace.
        
    - :ref:`librosa.decompose <decompose>`
        Functions for harmonic-percussive source separation (HPSS) and generic
        spectrogram decomposition using matrix decomposition methods implemented in
        *scikit-learn*.

    - :ref:`librosa.display <display>`
        Visualization and display routines using `matplotlib`.  

    - :ref:`librosa.effects <effects>`
        Time-domain audio processing, such as pitch shifting and time stretching.
        This submodule also provides time-domain wrappers for the `decompose`
        submodule.

    - :ref:`librosa.feature <feature>`
        Feature extraction and manipulation.  This includes low-level feature
        extraction, such as chromagrams, pseudo-constant-Q (log-frequency) transforms,
        Mel spectrogram, MFCC, and tuning estimation.  Also provided are feature
        manipulation methods, such as delta features, memory embedding, and
        event-synchronous feature alignment.

    - :ref:`librosa.filters <filters>`
        Filter-bank generation (chroma, pseudo-CQT, CQT, etc.).  These are primarily
        internal functions used by other parts of *librosa*.

    - :ref:`librosa.onset <onset>`
        Onset detection and onset strength computation.

    - :ref:`librosa.output <output>`
        Text- and wav-file output.

    - :ref:`librosa.segment <segment>`
        Functions useful for structural segmentation, such as recurrence matrix
        construction, time-lag representation, and sequentially constrained
        clustering.

    - :ref:`librosa.util <util>`
        Helper utilities (normalization, padding, centering, etc.)


.. _quickstart:

Quickstart
----------
Before diving into the details, we'll walk through a brief example program

.. code-block:: python
    :linenos:

    # Beat tracking example
    import librosa

    # 1. Get the file path to the included audio example
    filename = librosa.util.example_audio_file()

    # 2. Load the audio as a waveform `y`
    #    Store the sampling rate as `sr`
    y, sr = librosa.load(filename)

    # 3. Run the default beat tracker, using a hop length of 64 frames
    #    (64 frames at sr=22.050KHz ~= 2.9ms)
    hop_length = 64
    tempo, beat_frames = librosa.beat.beat_track(y=y, sr=sr, hop_length=hop_length)

    print 'Estimated tempo: %0.2f beats per minute' % tempo

    # 4. Convert the frame indices of beat events into timestamps
    beat_times = librosa.frames_to_time(beat_frames, sr=sr, hop_length=hop_length)

    print 'Saving output to beat_times.csv'
    librosa.output.times_csv('beat_times.csv', beat_times)


The first step of the program::

    filename = librosa.util.example_audio_file()

gets the path to the audio example file included with *librosa*.  After this step,
``filename`` will be a string variable containing the path to the example mp3.

The second step::

    y, sr = librosa.load(filename)
    
loads and decodes the audio as a :term:`time series` ``y``, represented as a one-dimensional
NumPy floating point array.  The variable ``sr`` contains the :term:`sampling rate` of
``y``, that is, the number of samples per second of audio.  By default, all audio is
mixed to mono and resampled to 22050 Hz at load time.  This behavior can be overridden
by supplying additional arguments to ``librosa.load()``.

The next line::

    hop_length = 64

sets the :term:`hop length` for the subsequent analysis.  This is number of samples to
advance between subsequent audio frames.  Here, we've set the hop length to 64
samples, which at 22KHz, comes to ``64.0 / 22050 ~= 2.9ms``.  

Next, we run the beat tracker using the specified hop length::

    tempo, beat_frames = librosa.beat.beat_track(y=y, sr=sr, hop_length=hop_length)

The output of the beat tracker is an estimate of the tempo (in beats per minute), 
and an array of frame numbers corresponding to detected beat events.

:term:`Frames <frame>` here correspond to short windows of the signal (``y``), each 
separated by ``hop_length`` samples.  Since v0.3, *librosa* uses centered frames, so 
that the *k*\ th frame is centered around sample ``k * hop_length``.

The next operation converts the frame numbers ``beat_frames`` into timings::

    beat_times = librosa.frames_to_time(beat_frames, sr=sr, hop_length=hop_length)

Now, ``beat_times`` will be an array of timestamps (in seconds) corresponding to
detected beat events.

Finally, we can store the detected beat timestamps as a comma-separated values (CSV)
file::

    librosa.output.times_csv('beat_times.csv', beat_times)

The contents of ``beat_times.csv`` should look like this::

    0.067
    0.514
    0.990
    1.454
    1.910
    ...

This is primarily useful for visualization purposes (e.g., using 
`Sonic Visualiser <http://www.sonicvisualiser.org>`_) or evaluation (e.g., using
`mir_eval <https://github.com/craffel/mir_eval>`_).


Advanced usage
--------------

Here we'll cover a more advanced example, integrating harmonic-percussive separation,
multiple spectral features, and beat-synchronous feature aggregation.

.. code-block:: python
    :linenos:

    # Feature extraction example

    import numpy as np
    import librosa

    # Load the example clip
    y, sr = librosa.load(librosa.util.example_audio_file())

    # Separate harmonics and percussives into two waveforms
    y_harmonic, y_percussive = librosa.effects.hpss(y)

    # Set the hop length
    hop_length = 64

    # Beat track on the percussive signal
    tempo, beat_frames = librosa.beat.beat_track(y=y_percussive, 
                                                 sr=sr,
                                                 hop_length=hop_length)

    # Compute MFCC features from the raw signal
    mfcc = librosa.feature.mfcc(y=y, sr=sr, hop_length=hop_length, n_mfcc=13)

    # And the first-order differences (delta features)
    mfcc_delta = librosa.feature.delta(mfcc)

    # Stack and synchronize between beat events
    # This time, we'll use the mean value (default) instead of median
    beat_mfcc_delta = librosa.feature.sync(np.vstack([mfcc, mfcc_delta]),
                                           beat_frames)

    # Compute chroma features from the harmonic signal
    chromagram = librosa.feature.chromagram(y=y_harmonic, 
                                            sr=sr,
                                            hop_length=hop_length)

    # Aggregate chroma features between beat events
    # We'll use the median value of each feature between beat frames
    beat_chroma = librosa.feature.sync(chromagram, 
                                       beat_frames,
                                       aggregate=np.median)

    # Finally, stack all beat-synchronous features together
    beat_features = np.vstack([beat_chroma, beat_mfcc_delta])


This example builds on tools we've already covered in the :ref:`quickstart example
<quickstart>`, so here we'll focus just on the new parts.

The first difference is the use of the :ref:`effects module <effects>` for time-series
harmonic-percussive separation::

    y_harmonic, y_percussive = librosa.effects.hpss(y)

The result of this line is that the time series ``y`` has been separated into two time
series, containing the harmonic (tonal) and percussive (transient) portions of the
signal.  Each of ``y_harmonic`` and ``y_percussive`` have the same shape and duration 
as ``y``.

The motivation for this kind of operation is two-fold: first, percussive elements
tend to be stronger indicators of rhythmic content, and can help provide more stable
beat tracking results; second, percussive elements can pollute tonal feature
representations (such as chroma) by contributing energy across all frequency bands, so
we'd be better off without them.

Next, we introduce the :ref:`feature module <feature>` and extract the Mel-frequency
cepstral coefficients from the raw signal ``y``::

    mfcc = librosa.feature.mfcc(y=y, sr=sr, hop_length=hop_length, n_mfcc=13)

The output of this function is the matrix ``mfcc``, which is an *numpy.ndarray* of
size ``(n_mfcc, T)`` (where ``T`` denotes the track duration in frames).  Note that we 
use the same ``hop_length`` here as in the beat tracker, so the detected ``beat_frames`` 
values correspond to columns of ``mfcc``.

The first type of feature manipulation we introduce is ``delta``, which computes
(smoothed) first-order differences among columns of its input::

    mfcc_delta = librosa.feature.delta(mfcc)

The resulting matrix ``mfcc_delta`` has the same shape as the input ``mfcc``.

The second type of feature manipulation is ``sync``, which aggregates columns of its
input between sample indices (e.g., beat frames)::

    beat_mfcc_delta = librosa.feature.sync(np.vstack([mfcc, mfcc_delta]),
                                           beat_frames)

Here, we've vertically stacked the ``mfcc`` and ``mfcc_delta`` matrices together.  The
result of this operation is a matrix ``beat_mfcc_delta`` with the same number of rows
as its input, but the number of columns depends on ``beat_frames``.  Each column 
``beat_mfcc_delta[:, k]`` will be the *average* of input columns between
``beat_frames[k]`` and ``beat_frames[k+1]``.  (``beat_frames`` will be expanded to
span the full range ``[0, T]`` so that all data is accounted for.)

Next, we compute a chromagram using just the harmonic component::

    chromagram = librosa.feature.chromagram(y=y_harmonic, 
                                            sr=sr,
                                            hop_length=hop_length)

After this line, ``chromagram`` will be a *numpy.ndarray* of size ``(12, T)``, and 
each row corresponds to a pitch class (e.g., *C*, *C#*, etc.).  Each column of 
``chromagram`` is normalized by its peak value, though this behavior can be overridden 
by setting the ``norm`` parameter.

Once we have the chromagram and list of beat frames, we again synchronize the chroma 
between beat events::

    beat_chroma = librosa.feature.sync(chromagram, 
                                       beat_frames,
                                       aggregate=np.median)

This time, we've replaced the default aggregate operation (*average*, as used above
for MFCCs) with the *median*.  In general, any statistical summarization function can
be supplied here, including `np.max()`, `np.min()`, `np.std()`, etc.

Finally, the all features are vertically stacked again::

    beat_features = np.vstack([beat_chroma, beat_mfcc_delta])

resulting in a feature matrix ``beat_features`` of dimension 
``(12 + 13 + 13, # beat intervals)``.


More examples
-------------

More example scripts are provided in the `examples
<https://github.com/bmcfee/librosa/tree/master/examples>`_ folder.