Output Sample Type and Sample Rate¶

• Output Sample Type and Sample Rate
  • Who should read this document
  • Rules of Thumb for Plugin Developers
  • Introduction
  • OneSamplePerStep
    • Examples
  • VariableSampleRate
    • Timestamps
    • Durations
    • Sample rate and "minimal" duration
  • FixedSampleRate
    • Timestamps
    • Examples

Who should read this document¶

This is a detailed document about the "sample type" and "sample rate" properties of a Vamp plugin's output descriptor.

• If you are new to the Vamp plugin API, read the Programmer's Guide first. The section "Sample Types and Timestamps" starting on page 9 introduces this subject.

• If you are writing a plugin, read the "Rules of Thumb" section (below) after the Programmer's Guide. You probably won't need to read the rest of this document. You should use the Vamp Plugin Tester to test your plugin.

• If you are writing a host, you should probably read the whole of this as well as the Guide. You should also use the Vamp Test Plugin to test your host's interpretation of the feature structures.

Rules of Thumb for Plugin Developers¶

The tl;dr summary:

• If your output returns things that are always regularly-spaced in time, and there is one such thing returned for every process block, and the calculation is causal so that results are available immediately, and there is no latency added beyond the length of the processing block, then you probably want to use OneSamplePerStep sample type and omit the feature timestamps.

• If your output returns things that are regularly-spaced in time but the other limitations above are not true, use FixedSampleRate sample type, set the output sample rate to the (perhaps fractional) number of returned features per second, and use a timestamp for each feature.

• If your output returns anything else, use VariableSampleRate sample type, set the output sample rate to zero unless you know better, and use a timestamp for each feature.

Introduction¶

A Vamp plugin receives audio and produces a series of descriptive feature structures.

The audio input is provided as a series of fixed-length sample blocks, equally spaced in time, provided to successive calls to the plugin's process function. The plugin may return any number of features from each process call, and may also return any number of features from getRemainingFeatures after all the audio has been received.

Features are each associated with a particular output of the plugin. The plugin declares that each output has certain properties, which constrain the sort of feature data the host can expect to see. (See diagram.)

A feature may or may not have a timestamp (as well as, optionally, a duration). Whether a timestamp is needed -- and, if it is provided, what it means -- are determined by the sampleType and sampleRate properties of the output on which the feature is returned.

An output's sampleType property may be either OneSamplePerStep, FixedSampleRate, or VariableSampleRate. Here's what they mean.

OneSamplePerStep¶

This is the simplest option.

If an output is declared as having a sampleType of OneSamplePerStep, then any features returned from a process call are assumed to match up with the audio block provided to that process call.

The sampleRate and hasDuration output properties are ignored for outputs of this type.

For any features returned through an output declared with OneSamplePerStep type,

• The plugin should not set timestamps on these features and should set their hasTimestamp property to false;
• The plugin should not set durations on these features and should set their hasDuration property to false;
• The host must ignore any timestamps or durations that the plugin may set on these features;
• The host must treat all such features returned from a given process call as if they had the same timestamp as it passed to that process call;
• The host must treat all such features returned from getRemainingFeatures as if they were immediately following the final process block (i.e. with the same time as the next equally-spaced process block would have had if the input had not ended);
• The host must treat all such features has having duration equal to the spacing between process blocks.

Examples¶

OneSamplePerStep is most often used for simple measurements and visualisations, in which some internal calculation is updated on each process call and a new result returned. For example: envelope trackers; power calculations; spectrograms. These outputs are typically visualised using line graphs or colour matrix plots.

OneSamplePerStep is often used for intermediate results calculated during processing of a more sophisticated feature. For example, a beat tracker might have an auxiliary output with OneSamplePerStep type returning its internal onset detection function value.

VariableSampleRate¶

If the OneSamplePerStep output type essentially means that the plugin leaves all time calculations up to the host, VariableSampleRate is the opposite.

If an output is declared as having a SampleType of VariableSampleRate, the features returned through it will have timestamps set by the plugin, and they won't necessarily have any relationship to the process block timestamps provided by the host.

Timestamps¶

For any features returned through an output declared with VariableSampleRate type,

• The plugin must set timestamps on these features and must set their hasTimestamp property to true;
• The host must obtain the features' start times from their timestamps rather than calculating them itself.

Durations¶

Features returned through VariableSampleRate outputs may optionally have durations.

If the output's hasDuration property is true, then

• The plugin may set the hasDuration property of features returned through this output to true and, if it does so, must also set their duration property;
• If a feature's hasDuration property is true, then the host must use the feature's duration property as the feature duration; otherwise the host must treat the feature as having "minimal" duration (see "Sample Rate" below).

If the output's hasDuration property is false, then

• The plugin should not set the duration property of that output's features;
• The host must ignore the hasDuration and duration properties of the output's features, and treat them as having "minimal" duration (see below).

Sample rate and "minimal" duration¶

The plugin may optionally set a sampleRate property for each VariableSampleRate output. A sampleRate of zero indicates no value.

If a sampleRate is set,

• The host must use 1/sampleRate seconds as the "minimal" duration assigned to features that have no duration supplied.

If no sampleRate is set, i.e. if the sampleRate property is zero,

• The host must use zero as the "minimal" duration used for features with no duration supplied.

FixedSampleRate¶

This is a more complex case than OneSamplePerStep or VariableSampleRate. A OneSamplePerStep output never uses individual feature timestamps; a VariableSampleRate output always uses them; a FixedSampleRate output sometimes does.

Timestamps¶

An output with FixedSampleRate type must also have a non-zero sampleRate property.

Conceptually the sampleRate property defines a set of discrete sample times, spaced at sampleRate points per second, starting at time zero. Every returned feature will have one of these times, whether it is assigned by the plugin (through the timestamp of the feature) or by the host (for features without timestamps).

For any features returned through an output declared with FixedSampleRate type,

• The plugin may set timestamps on these features, and it must set the feature's corresponding hasTimestamp property to true or false depending on whether it has set a timestamp;
• Where a feature has a timestamp, the host must take that timestamp, round it to the nearest 1/sampleRate seconds, and use that as the time of the feature;
• Where a feature has no timestamp, the host must take the time of the previous feature, add 1/sampleRate seconds to it, and use that as the time of the new feature;
• Where the very first feature has no timestamp, the host must take its time as zero.

The Programmer's Guide leaves this behaviour incompletely defined, saying "The host may round the timestamp according to the sample rate given in the output descriptor's sampleRate field". Here we are declaring that the host must round the timestamp in this way. Note that the Vamp Test Plugin assumes this behaviour.

Examples¶