SampleType » History » Version 33

Chris Cannam, 2014-02-10 03:16 PM

1 1 Chris Cannam
h1. Output Sample Type and Sample Rate
2 1 Chris Cannam
3 15 Chris Cannam
{{>toc}}
4 15 Chris Cannam
5 30 Chris Cannam
h2. Who should read this document
6 29 Chris Cannam
7 30 Chris Cannam
This is a detailed document about the "sample type" and "sample rate" properties of a Vamp plugin's output descriptor.
8 29 Chris Cannam
9 29 Chris Cannam
 * If you are new to the Vamp plugin API, read the "Programmer's Guide":http://vamp-plugins.org/guide.pdf first. The section "Sample Types and Timestamps" starting on page 9 introduces this subject.
10 29 Chris Cannam
11 31 Chris Cannam
 * If you are writing a plugin, read the "Rules of Thumb" section (below) after the Programmer's Guide. You probably won't need to read the rest of this document. You should use the "Vamp Plugin Tester":/projects/vamp-plugin-tester to test your plugin.
12 29 Chris Cannam
13 29 Chris Cannam
 * If you are writing a host, you should probably read the whole of this as well as the Guide. You should also use the "Vamp Test Plugin":/projects/vamp-test-plugin to test your host's interpretation of the feature structures.
14 29 Chris Cannam
15 29 Chris Cannam
h2. Rules of Thumb for Plugin Developers
16 29 Chris Cannam
17 29 Chris Cannam
The tl;dr summary:
18 29 Chris Cannam
19 32 Chris Cannam
 * If your output returns things that are always regularly-spaced in time, and there is one such thing returned for every @process@ block, and the calculation is causal so that results are available immediately, you probably want to use @OneSamplePerStep@ sample type and omit the feature timestamps.
20 29 Chris Cannam
21 29 Chris Cannam
 * If your output returns things that are regularly-spaced in time but the other limitations above are not met, use @FixedSampleRate@ sample type, set the output sample rate to the (perhaps fractional) number of returned features per second, and use a timestamp for each feature.
22 29 Chris Cannam
23 1 Chris Cannam
 * If your output returns anything else, use @VariableSampleRate@ sample type, set the output sample rate to zero unless you know better, and use a timestamp for each feature.
24 32 Chris Cannam
25 29 Chris Cannam
26 29 Chris Cannam
h2. Introduction
27 29 Chris Cannam
28 12 Chris Cannam
A Vamp plugin receives audio and produces a series of descriptive feature structures.
29 1 Chris Cannam
30 12 Chris Cannam
The audio input is provided as a series of fixed-length sample blocks, equally spaced in time, provided to successive calls to the plugin's @process@ function. The plugin may return any number of features from each @process@ call, and may also return any number of features from @getRemainingFeatures@ after all the audio has been received.
31 11 Chris Cannam
32 11 Chris Cannam
Features are each associated with a particular output of the plugin. The plugin declares that each output has certain properties, which constrain the sort of feature data the host can expect to see. (See diagram.)
33 10 Chris Cannam
34 7 Chris Cannam
!/attachments/download/980/feature-structures-20pc.png!
35 9 Chris Cannam
36 22 Chris Cannam
A feature may or may not have a timestamp (as well as, optionally, a duration). Whether a timestamp is needed -- and, if it is provided, what it means -- are determined by the @sampleType@ and @sampleRate@ properties of the output on which the feature is returned.
37 7 Chris Cannam
38 27 Chris Cannam
An output's @sampleType@ property may be either @OneSamplePerStep@, @FixedSampleRate@, or @VariableSampleRate@. Here's what they mean.
39 1 Chris Cannam
40 27 Chris Cannam
h2. OneSamplePerStep
41 1 Chris Cannam
42 23 Chris Cannam
This is the simplest option.
43 1 Chris Cannam
44 23 Chris Cannam
If an output is declared as having a @sampleType@ of @OneSamplePerStep@, then any features returned from a @process@ call are assumed to match up with the audio block provided to that @process@ call.
45 23 Chris Cannam
46 14 Chris Cannam
The @sampleRate@ and @hasDuration@ output properties are ignored for outputs of this type.
47 1 Chris Cannam
48 14 Chris Cannam
For any features returned through an output declared with @OneSamplePerStep@ type,
49 14 Chris Cannam
50 24 Chris Cannam
 * The plugin _should not_ set timestamps on these features and _should_ set their @hasTimestamp@ property to @false@;
51 1 Chris Cannam
 * The plugin _should not_ set durations on these features and _should_ set their @hasDuration@ property to @false@;
52 33 Chris Cannam
 * The host _must_ ignore any timestamps or durations that the plugin may set on these features;
53 24 Chris Cannam
 * The host _must_ treat all such features returned from a given @process@ call as if they had the same timestamp as it passed to that @process@ call;
54 1 Chris Cannam
 * The host _must_ treat all such features returned from @getRemainingFeatures@ as if they were immediately following the final @process@ block (i.e. with the same time as the next equally-spaced @process@ block would have had if the input had not ended);
55 22 Chris Cannam
 * The host _must_ treat all such features has having duration equal to the spacing between process blocks.
56 1 Chris Cannam
57 27 Chris Cannam
h3. Examples
58 14 Chris Cannam
59 14 Chris Cannam
@OneSamplePerStep@ is most often used for simple measurements and visualisations, in which some internal calculation is updated on each process call and a new result returned. For example: envelope trackers; power calculations; spectrograms. These outputs are typically visualised using line graphs or colour matrix plots.
60 14 Chris Cannam
61 1 Chris Cannam
@OneSamplePerStep@ is often used for intermediate results calculated during processing of a more sophisticated feature. For example, a beat tracker might have an auxiliary output with @OneSamplePerStep@ type returning its internal onset detection function value.
62 14 Chris Cannam
63 27 Chris Cannam
h2. VariableSampleRate
64 20 Chris Cannam
65 21 Chris Cannam
If the @OneSamplePerStep@ output type essentially means that the plugin leaves all time calculations up to the host, @VariableSampleRate@ is the opposite.
66 22 Chris Cannam
67 1 Chris Cannam
If an output is declared as having a @SampleType@ of @VariableSampleRate@, the features returned through it will have timestamps set by the plugin, and they won't necessarily have any relationship to the process block timestamps provided by the host.
68 1 Chris Cannam
69 27 Chris Cannam
h3. Timestamps
70 25 Chris Cannam
71 1 Chris Cannam
For any features returned through an output declared with @VariableSampleRate@ type,
72 24 Chris Cannam
73 24 Chris Cannam
 * The plugin _must_ set timestamps on these features and _must_ set their @hasTimestamp@ property to @true@;
74 28 Chris Cannam
 * The host _must_ obtain the features' start times from their timestamps rather than calculating them itself.
75 25 Chris Cannam
76 27 Chris Cannam
h3. Durations
77 25 Chris Cannam
78 1 Chris Cannam
Features returned through @VariableSampleRate@ outputs may optionally have durations.
79 25 Chris Cannam
80 26 Chris Cannam
If the output's @hasDuration@ property is @true@, then
81 26 Chris Cannam
82 28 Chris Cannam
 * The plugin _may_ set the @hasDuration@ property of such features to @true@ and, if it does so, _must_ also set their @duration@ property;
83 28 Chris Cannam
 * If a feature's @hasDuration@ property is true, then the host _must_ use the feature's @duration@ property as the feature duration; otherwise the host _must_ treat the feature as having "minimal" duration (see "Sample Rate" below).
84 1 Chris Cannam
85 28 Chris Cannam
If the output's @hasDuration@ property is @false@, then
86 1 Chris Cannam
87 28 Chris Cannam
 * The plugin _should not_ set the @duration@ property of that output's features;
88 28 Chris Cannam
 * The host _must_ ignore the @hasDuration@ and @duration@ properties of the features and treat them as having "minimal" duration (see below).
89 28 Chris Cannam
90 28 Chris Cannam
h3. Sample rate and "minimal" duration
91 28 Chris Cannam
92 28 Chris Cannam
The plugin may optionally set a @sampleRate@ property for each @VariableSampleRate@ output. A @sampleRate@ of zero indicates no value.
93 28 Chris Cannam
94 28 Chris Cannam
If a @sampleRate@ is set,
95 28 Chris Cannam
96 28 Chris Cannam
 * The host _may_ optionally use the 1/@sampleRate@ seconds as indicating the resolution of the output feature timestamps, and _may_ round each output feature timestamp to a multiple of that resolution;
97 28 Chris Cannam
 * The host _must_ use 1/@sampleRate@ seconds as the "minimal" duration assigned to features that have no duration supplied.
98 28 Chris Cannam
99 28 Chris Cannam
If no @sampleRate@ is set,
100 28 Chris Cannam
101 28 Chris Cannam
 * The host _must_ use the feature timestamps unmodified;
102 28 Chris Cannam
 * The host must use zero as the "minimal" duration used for features with no duration supplied.
103 22 Chris Cannam
104 27 Chris Cannam
h3. Examples
105 1 Chris Cannam
106 27 Chris Cannam
h2. FixedSampleRate
107 22 Chris Cannam
108 22 Chris Cannam
If an output is declared as having a @SampleType@ of @FixedSampleRate@
109 22 Chris Cannam
110 27 Chris Cannam
h3. Examples