Desirable goals » History » Version 4

Chris Cannam, 2015-01-12 12:19 PM

1 1 Chris Cannam
h1. Desirable goals
2 1 Chris Cannam
3 3 Chris Cannam
*Dan writes:*
4 1 Chris Cannam
5 2 Chris Cannam
_I would request that a function returning data block-by-block should be a "generator":https://wiki.python.org/moin/Generators which would make
6 2 Chris Cannam
it very flexible, and could also be used to avoid having to have all the data in memory at once._
7 2 Chris Cannam
8 2 Chris Cannam
...
9 2 Chris Cannam
10 2 Chris Cannam
_from my point of view, and not having thought through all the flexible types of output that vamp provides, I'd be hoping to type things like:_
11 1 Chris Cannam
12 1 Chris Cannam
<pre>
13 1 Chris Cannam
# all in-memory:
14 1 Chris Cannam
15 1 Chris Cannam
plug = vh.loadPlugin('vamp-example-plugins:mfcc', 44100)
16 1 Chris Cannam
mfccs = np.array([features[0] for features in plug.processall([-0.1, 0, 0.1, 0, -0.1, 0, 0.1, 0])])  # this gives a 2D numpy array of shape [nframes, nmfccs]
17 1 Chris Cannam
18 1 Chris Cannam
plug = vh.loadPlugin('vamp-example-plugins:onsetdetector', 44100)
19 1 Chris Cannam
onsets = np.array([features[0] for features in plug.processall([-0.1, 0, 0.1, 0, -0.1, 0, 0.1, 0])])  # this gives a 1D numpy array, a list of onset times I guess
20 1 Chris Cannam
21 1 Chris Cannam
22 1 Chris Cannam
# from disk, to memory:
23 1 Chris Cannam
24 1 Chris Cannam
plug = vh.loadPlugin('vamp-example-plugins:mfcc', 44100)
25 1 Chris Cannam
with open(filepath) as f:
26 1 Chris Cannam
	mfccs = np.array([features[0] for features in plug.processall(f)])
27 1 Chris Cannam
plt.matshow(mfccs, interpolate='nearest') # a simple pyplot
28 1 Chris Cannam
29 1 Chris Cannam
30 1 Chris Cannam
# fully streaming:
31 1 Chris Cannam
32 1 Chris Cannam
plug = vh.loadPlugin('vamp-example-plugins:mfcc', 44100)
33 1 Chris Cannam
with open(filepath) as f:
34 1 Chris Cannam
	with open(outpath, 'wb') as outf:
35 1 Chris Cannam
		for features in plug.processall(f):
36 1 Chris Cannam
			outpath.write("%g\n" % features[0][0] ** 2)
37 1 Chris Cannam
</pre>
38 1 Chris Cannam
39 1 Chris Cannam
_BTW I noticed I made a couple of mistakes in there_
40 3 Chris Cannam
41 3 Chris Cannam
...
42 3 Chris Cannam
43 3 Chris Cannam
*Chris writes:*
44 3 Chris Cannam
45 3 Chris Cannam
API-wise, that's slightly problematic because the standard plugin interface goes
46 3 Chris Cannam
47 3 Chris Cannam
<pre>
48 3 Chris Cannam
  load -> initialise -> process, process, process
49 3 Chris Cannam
</pre>
50 3 Chris Cannam
51 3 Chris Cannam
and one of the biggest problems is people forgetting / misunderstanding the initialise call. (Ignoring for now the merits or otherwise of the API design -- I just mean that this is the way it works now)
52 3 Chris Cannam
53 3 Chris Cannam
So introducing another workflow that looks like
54 3 Chris Cannam
55 3 Chris Cannam
<pre>
56 3 Chris Cannam
  load -> processall
57 3 Chris Cannam
</pre>
58 3 Chris Cannam
59 3 Chris Cannam
might be hazardous from that perspective.
60 3 Chris Cannam
61 3 Chris Cannam
*Dan:*
62 3 Chris Cannam
63 3 Chris Cannam
Note that the generator style means that it's still possible to do "process, process, process" if needed, for example:
64 3 Chris Cannam
65 3 Chris Cannam
<pre>
66 3 Chris Cannam
 mygenerator = plug.processall(f)
67 3 Chris Cannam
 print mygenerator.next()   # frame 0?
68 3 Chris Cannam
 print mygenerator.next()   # frame 1?
69 3 Chris Cannam
 print mygenerator.next()   # frame 2?
70 3 Chris Cannam
</pre>
71 3 Chris Cannam
72 3 Chris Cannam
and the initialisation stuff is still bound up in there neatly rather than being forgettable.
73 3 Chris Cannam
74 3 Chris Cannam
*Chris:*
75 3 Chris Cannam
76 3 Chris Cannam
Also of course if you're running it on an audio file, it knows the sample rate / channel count etc so you should be able to short-circuit
77 3 Chris Cannam
it:
78 3 Chris Cannam
79 3 Chris Cannam
<pre>
80 3 Chris Cannam
   import vampyhost as vh
81 3 Chris Cannam
   results = vh.processall(file, 'vamp-example-plugins:zerocrossings')
82 3 Chris Cannam
</pre>
83 3 Chris Cannam
84 3 Chris Cannam
and it loads (& initialises) the plugin for you.
85 3 Chris Cannam
86 3 Chris Cannam
*Dan:*
87 3 Chris Cannam
88 3 Chris Cannam
Yes, feels good as high-level api
89 3 Chris Cannam
90 3 Chris Cannam
*Chris:*
91 3 Chris Cannam
92 4 Chris Cannam
Another advantage of making processall (or whatever it's called) callable through the module rather than the plugin object is that it ensures you don't go and use the plugin for something else afterwards (with unpredictable results).
93 3 Chris Cannam
94 4 Chris Cannam
Trouble is if you're running it on array data, you have to supply the rate somewhere so you must either load() explicitly or provide the rate to processall.
95 3 Chris Cannam
96 4 Chris Cannam
(I am very keen to make simple use cases like "give me the beat times in this audio file" as simple as possible. Of course plenty of other APIs can now be used to do that, but that doesn't mean we shouldn't do the best job possible with this one)