Convert a Vamp Plugin to a JS Module » History » Version 4

Chris Cannam, 2017-06-20 10:17 AM

1 1 Chris Cannam
h1. Convert a Vamp Plugin to a JS Module
2 1 Chris Cannam
3 1 Chris Cannam
This page will describe how to take an existing Vamp plugin library, in the form of C++/C source code, and recompile it into a Javascript module that provides the "Piper":/projects/piper interface.
4 1 Chris Cannam
5 1 Chris Cannam
The process is a bit tricky, but it should only have to be done once for each plugin library, as the Javascript build is portable (unlike native builds of the original plugins which have to be re-done for each platform).
6 1 Chris Cannam
7 1 Chris Cannam
The process is also complicated by any third-party library code that the plugin may use. It's not possible to take an existing native library file (whether .a, .lib, .so, .dylib or .dll) and convert that to Javascript -- you have to add the original library source files to your build. This could be tricky to do if the library has any particular configuration requirements.
8 2 Chris Cannam
9 4 Chris Cannam
h4. Examples
10 4 Chris Cannam
11 4 Chris Cannam
Examples of plugins that have already been converted can be found in the @examples@ directory in the @piper-vamp-js@ repository. The @vamp-example-plugins@ example is the most complete.
12 4 Chris Cannam
13 3 Chris Cannam
h4. An illustrative Docker file
14 2 Chris Cannam
15 2 Chris Cannam
To accompany this explanation, you can "find an example Docker file here":https://github.com/piper-audio/piper-vamp-js/blob/master/examples/docker/Dockerfile which actually carries out the process of converting and building a plugin library within a Docker container. This is not an especially convenient way to do it, but it should make explicit all of the necessary steps in a more-or-less reproducible way.
16 2 Chris Cannam
17 2 Chris Cannam
h4. Prerequisites
18 2 Chris Cannam
19 2 Chris Cannam
You will need the following:
20 2 Chris Cannam
21 2 Chris Cannam
* A typical Unix-like system - this has been tested using Arch Linux and macOS
22 2 Chris Cannam
* A sufficiently recent version of the Emscripten C++-to-Javascript compiler
23 2 Chris Cannam
* Typical native C/C++ build tools
24 2 Chris Cannam
* Node.js environment, to run the tests
25 2 Chris Cannam
26 2 Chris Cannam
You will need to have the following repositories checked out. The build process expects that all of these will be checked out into a common parent directory.
27 2 Chris Cannam
28 2 Chris Cannam
* "Piper":/projects/piper - basic schema
29 2 Chris Cannam
* "Piper Vamp C++":/projects/piper-cpp - supporting C++ code
30 2 Chris Cannam
* "Piper Vamp JS":/projects/piper-vamp-js - Javascript adapter code
31 2 Chris Cannam
* "Vamp Plugin SDK":/projects/vamp-plugin-sdk - Vamp SDK used by both of the last two
32 3 Chris Cannam
* The source code for your own plugin library which you intend to convert
33 1 Chris Cannam
34 1 Chris Cannam
Also create a new directory for the JS module to be compiled in. You will need to provide a Makefile and a main file (in C++, to be cross-compiled to JS along with everything else) for this module so it makes sense to have a dedicated directory (and likely a version control repository) to put them in.
35 3 Chris Cannam
36 3 Chris Cannam
h4. Build the dependencies and tools
37 3 Chris Cannam
38 3 Chris Cannam
Not all of the dependent repositories need to be compiled: the ones that do are the Vamp Plugin SDK and Piper Vamp JS.
39 3 Chris Cannam
40 3 Chris Cannam
Build and install the Vamp plugin SDK. On Linux this is
41 3 Chris Cannam
42 3 Chris Cannam
<pre>
43 3 Chris Cannam
$ cd vamp-plugin-sdk ; ./configure && make && sudo make install
44 3 Chris Cannam
</pre>
45 3 Chris Cannam
46 3 Chris Cannam
Build the Piper Vamp JS repository:
47 3 Chris Cannam
48 3 Chris Cannam
<pre>
49 3 Chris Cannam
$ cd piper-vamp-js ; make
50 3 Chris Cannam
</pre>
51 3 Chris Cannam
52 3 Chris Cannam
This should produce a generator program in @bin/piper-vamp-stub-generator@ which we will use in a moment.
53 3 Chris Cannam
54 3 Chris Cannam
h4. Prepare your own plugin library
55 3 Chris Cannam
56 3 Chris Cannam
Compile your own Vamp plugin library in the normal way, i.e. as a native Vamp plugin for your current platform. This is necessary so that the conversion stub generator program can query the plugin.
57 3 Chris Cannam
58 3 Chris Cannam
Add its location to the @VAMP_PATH@ environment variable. You should also check that the @.cat@ category file and the @.n3@ or @.ttl@ RDF description file for the plugin are available in the same location as the plugin library itself. (If the plugin doesn't have those files, consider making them? They're helpful for other Vamp plugin hosts as well.)
59 3 Chris Cannam
60 3 Chris Cannam
h4. Prepare the JS module code folder
61 3 Chris Cannam
62 3 Chris Cannam
In your new working directory for the converted code, generate an initial version of the module main entry point using the generator program in @piper-vamp-js@. This will only work if your plugin library is in the @VAMP_PATH@.
63 3 Chris Cannam
64 3 Chris Cannam
<pre>
65 3 Chris Cannam
$ ../piper-vamp-js/bin/piper-vamp-stub-generator plugin-library-name > plugin-library-name.cpp
66 3 Chris Cannam
</pre>
67 3 Chris Cannam
68 3 Chris Cannam
replacing @plugin-library-name@ with the name of your plugin library. Be sure to run this in the right directory - the (initially empty) one for your new JS module, not the original plugin directory (as that could overwrite the original plugin code).
69 4 Chris Cannam
70 4 Chris Cannam
h4. Fix the generated code and add a Makefile
71 4 Chris Cannam
72 4 Chris Cannam
Open the generated file (referred to above as @plugin-library-name.cpp@) in an editor and check its contents. The generator guesses the names of likely plugin header files and classes based on the plugin identifiers, and they are probably not actually correct. If your library is accompanied by valid category and RDF files, you should find suitable category and output type URI metadata for each plugin in the generated file -- if these are absent, check that the original plugin location has the appropriate files.
73 4 Chris Cannam
74 4 Chris Cannam
Now the part that needs the most actual work: create a Makefile to build your plugin code with Emscripten.
75 4 Chris Cannam
76 4 Chris Cannam
The @piper-vamp-js@ repository contains most of what you need, in the form of a file @Makefile.inc@ that you can include from your Makefile. But you need to provide all the plugin-specific source files and any additional build flags, and set the variables that @Makefile.inc@ expects. There is a comment at the top of @Makefile.inc@ that explains this further.
77 4 Chris Cannam
78 4 Chris Cannam
An example Makefile is as follows:
79 4 Chris Cannam
80 4 Chris Cannam
<pre>
81 4 Chris Cannam
PIPER_VAMP_JS_DIR       := ../piper-vamp-js
82 4 Chris Cannam
PLUGIN_DIR              := ../cepstral-pitchtracker
83 4 Chris Cannam
84 4 Chris Cannam
SRC_DIR                 := $(PLUGIN_DIR)
85 4 Chris Cannam
86 4 Chris Cannam
MODULE_NAME             := CepstralPitchTracker
87 4 Chris Cannam
MODULE_SOURCE           := cepstral-pitchtracker.cpp
88 4 Chris Cannam
89 4 Chris Cannam
PLUGIN_SOURCES          := $(SRC_DIR)/AgentFeeder.cpp \
90 4 Chris Cannam
                           $(SRC_DIR)/CepstralPitchTracker.cpp \
91 4 Chris Cannam
                           $(SRC_DIR)/NoteHypothesis.cpp \
92 4 Chris Cannam
                           $(SRC_DIR)/PeakInterpolator.cpp
93 4 Chris Cannam
94 4 Chris Cannam
INCLUDES                := -I$(SRC_DIR)
95 4 Chris Cannam
96 4 Chris Cannam
include $(PIPER_VAMP_JS_DIR)/Makefile.inc
97 4 Chris Cannam
</pre>
98 4 Chris Cannam
99 4 Chris Cannam
This goes into the same directory as your entry point file.
100 4 Chris Cannam
101 4 Chris Cannam
Now, to build the JS module, run
102 4 Chris Cannam
103 4 Chris Cannam
<pre>
104 4 Chris Cannam
$ make clean
105 4 Chris Cannam
$ make em
106 4 Chris Cannam
</pre>
107 4 Chris Cannam
108 4 Chris Cannam
This should produce a pair of JS files called (in this case) @CepstralPitchTracker.js@ and @CepstralPitchTracker.umd.js@. Both of these contain the same module code, just wrapped in different ways (plain Emscripten module vs UMD wrapper). Use whichever your application finds more convenient.
109 4 Chris Cannam
110 4 Chris Cannam
To test the resulting build, run
111 4 Chris Cannam
112 4 Chris Cannam
<pre>
113 4 Chris Cannam
$ make test
114 4 Chris Cannam
</pre>