Convert a Vamp Plugin to a JS Module » History » Version 4
Version 3 (Chris Cannam, 2017-06-20 10:04 AM) → Version 4/7 (Chris Cannam, 2017-06-20 10:17 AM)
h1. Convert a Vamp Plugin to a JS Module
This page will describe how to take an existing Vamp plugin library, in the form of C++/C source code, and recompile it into a Javascript module that provides the "Piper":/projects/piper interface.
The process is a bit tricky, but it should only have to be done once for each plugin library, as the Javascript build is portable (unlike native builds of the original plugins which have to be re-done for each platform).
The process is also complicated by any third-party library code that the plugin may use. It's not possible to take an existing native library file (whether .a, .lib, .so, .dylib or .dll) and convert that to Javascript -- you have to add the original library source files to your build. This could be tricky to do if the library has any particular configuration requirements.
h4. Examples
Examples of plugins that have already been converted can be found in the @examples@ directory in the @piper-vamp-js@ repository. The @vamp-example-plugins@ example is the most complete.
h4. An illustrative Docker file
To accompany this explanation, you can "find an example Docker file here":https://github.com/piper-audio/piper-vamp-js/blob/master/examples/docker/Dockerfile which actually carries out the process of converting and building a plugin library within a Docker container. This is not an especially convenient way to do it, but it should make explicit all of the necessary steps in a more-or-less reproducible way.
h4. Prerequisites
You will need the following:
* A typical Unix-like system - this has been tested using Arch Linux and macOS
* A sufficiently recent version of the Emscripten C++-to-Javascript compiler
* Typical native C/C++ build tools
* Node.js environment, to run the tests
You will need to have the following repositories checked out. The build process expects that all of these will be checked out into a common parent directory.
* "Piper":/projects/piper - basic schema
* "Piper Vamp C++":/projects/piper-cpp - supporting C++ code
* "Piper Vamp JS":/projects/piper-vamp-js - Javascript adapter code
* "Vamp Plugin SDK":/projects/vamp-plugin-sdk - Vamp SDK used by both of the last two
* The source code for your own plugin library which you intend to convert
Also create a new directory for the JS module to be compiled in. You will need to provide a Makefile and a main file (in C++, to be cross-compiled to JS along with everything else) for this module so it makes sense to have a dedicated directory (and likely a version control repository) to put them in.
h4. Build the dependencies and tools
Not all of the dependent repositories need to be compiled: the ones that do are the Vamp Plugin SDK and Piper Vamp JS.
Build and install the Vamp plugin SDK. On Linux this is
<pre>
$ cd vamp-plugin-sdk ; ./configure && make && sudo make install
</pre>
Build the Piper Vamp JS repository:
<pre>
$ cd piper-vamp-js ; make
</pre>
This should produce a generator program in @bin/piper-vamp-stub-generator@ which we will use in a moment.
h4. Prepare your own plugin library
Compile your own Vamp plugin library in the normal way, i.e. as a native Vamp plugin for your current platform. This is necessary so that the conversion stub generator program can query the plugin.
Add its location to the @VAMP_PATH@ environment variable. You should also check that the @.cat@ category file and the @.n3@ or @.ttl@ RDF description file for the plugin are available in the same location as the plugin library itself. (If the plugin doesn't have those files, consider making them? They're helpful for other Vamp plugin hosts as well.)
h4. Prepare the JS module code folder
In your new working directory for the converted code, generate an initial version of the module main entry point using the generator program in @piper-vamp-js@. This will only work if your plugin library is in the @VAMP_PATH@.
<pre>
$ ../piper-vamp-js/bin/piper-vamp-stub-generator plugin-library-name > plugin-library-name.cpp
</pre>
replacing @plugin-library-name@ with the name of your plugin library. Be sure to run this in the right directory - the (initially empty) one for your new JS module, not the original plugin directory (as that could overwrite the original plugin code).
h4. Fix the generated code and add a Makefile
Open the generated file (referred to above as @plugin-library-name.cpp@) in an editor and check its contents. The generator guesses the names of likely plugin header files and classes based on the plugin identifiers, and they are probably not actually correct. If your library is accompanied by valid category and RDF files, you should find suitable category and output type URI metadata for each plugin in the generated file -- if these are absent, check that the original plugin location has the appropriate files.
Now the part that needs the most actual work: create a Makefile to build your plugin code with Emscripten.
The @piper-vamp-js@ repository contains most of what you need, in the form of a file @Makefile.inc@ that you can include from your Makefile. But you need to provide all the plugin-specific source files and any additional build flags, and set the variables that @Makefile.inc@ expects. There is a comment at the top of @Makefile.inc@ that explains this further.
An example Makefile is as follows:
<pre>
PIPER_VAMP_JS_DIR := ../piper-vamp-js
PLUGIN_DIR := ../cepstral-pitchtracker
SRC_DIR := $(PLUGIN_DIR)
MODULE_NAME := CepstralPitchTracker
MODULE_SOURCE := cepstral-pitchtracker.cpp
PLUGIN_SOURCES := $(SRC_DIR)/AgentFeeder.cpp \
$(SRC_DIR)/CepstralPitchTracker.cpp \
$(SRC_DIR)/NoteHypothesis.cpp \
$(SRC_DIR)/PeakInterpolator.cpp
INCLUDES := -I$(SRC_DIR)
include $(PIPER_VAMP_JS_DIR)/Makefile.inc
</pre>
This goes into the same directory as your entry point file.
Now, to build the JS module, run
<pre>
$ make clean
$ make em
</pre>
This should produce a pair of JS files called (in this case) @CepstralPitchTracker.js@ and @CepstralPitchTracker.umd.js@. Both of these contain the same module code, just wrapped in different ways (plain Emscripten module vs UMD wrapper). Use whichever your application finds more convenient.
To test the resulting build, run
<pre>
$ make test
</pre>
This page will describe how to take an existing Vamp plugin library, in the form of C++/C source code, and recompile it into a Javascript module that provides the "Piper":/projects/piper interface.
The process is a bit tricky, but it should only have to be done once for each plugin library, as the Javascript build is portable (unlike native builds of the original plugins which have to be re-done for each platform).
The process is also complicated by any third-party library code that the plugin may use. It's not possible to take an existing native library file (whether .a, .lib, .so, .dylib or .dll) and convert that to Javascript -- you have to add the original library source files to your build. This could be tricky to do if the library has any particular configuration requirements.
h4. Examples
Examples of plugins that have already been converted can be found in the @examples@ directory in the @piper-vamp-js@ repository. The @vamp-example-plugins@ example is the most complete.
h4. An illustrative Docker file
To accompany this explanation, you can "find an example Docker file here":https://github.com/piper-audio/piper-vamp-js/blob/master/examples/docker/Dockerfile which actually carries out the process of converting and building a plugin library within a Docker container. This is not an especially convenient way to do it, but it should make explicit all of the necessary steps in a more-or-less reproducible way.
h4. Prerequisites
You will need the following:
* A typical Unix-like system - this has been tested using Arch Linux and macOS
* A sufficiently recent version of the Emscripten C++-to-Javascript compiler
* Typical native C/C++ build tools
* Node.js environment, to run the tests
You will need to have the following repositories checked out. The build process expects that all of these will be checked out into a common parent directory.
* "Piper":/projects/piper - basic schema
* "Piper Vamp C++":/projects/piper-cpp - supporting C++ code
* "Piper Vamp JS":/projects/piper-vamp-js - Javascript adapter code
* "Vamp Plugin SDK":/projects/vamp-plugin-sdk - Vamp SDK used by both of the last two
* The source code for your own plugin library which you intend to convert
Also create a new directory for the JS module to be compiled in. You will need to provide a Makefile and a main file (in C++, to be cross-compiled to JS along with everything else) for this module so it makes sense to have a dedicated directory (and likely a version control repository) to put them in.
h4. Build the dependencies and tools
Not all of the dependent repositories need to be compiled: the ones that do are the Vamp Plugin SDK and Piper Vamp JS.
Build and install the Vamp plugin SDK. On Linux this is
<pre>
$ cd vamp-plugin-sdk ; ./configure && make && sudo make install
</pre>
Build the Piper Vamp JS repository:
<pre>
$ cd piper-vamp-js ; make
</pre>
This should produce a generator program in @bin/piper-vamp-stub-generator@ which we will use in a moment.
h4. Prepare your own plugin library
Compile your own Vamp plugin library in the normal way, i.e. as a native Vamp plugin for your current platform. This is necessary so that the conversion stub generator program can query the plugin.
Add its location to the @VAMP_PATH@ environment variable. You should also check that the @.cat@ category file and the @.n3@ or @.ttl@ RDF description file for the plugin are available in the same location as the plugin library itself. (If the plugin doesn't have those files, consider making them? They're helpful for other Vamp plugin hosts as well.)
h4. Prepare the JS module code folder
In your new working directory for the converted code, generate an initial version of the module main entry point using the generator program in @piper-vamp-js@. This will only work if your plugin library is in the @VAMP_PATH@.
<pre>
$ ../piper-vamp-js/bin/piper-vamp-stub-generator plugin-library-name > plugin-library-name.cpp
</pre>
replacing @plugin-library-name@ with the name of your plugin library. Be sure to run this in the right directory - the (initially empty) one for your new JS module, not the original plugin directory (as that could overwrite the original plugin code).
h4. Fix the generated code and add a Makefile
Open the generated file (referred to above as @plugin-library-name.cpp@) in an editor and check its contents. The generator guesses the names of likely plugin header files and classes based on the plugin identifiers, and they are probably not actually correct. If your library is accompanied by valid category and RDF files, you should find suitable category and output type URI metadata for each plugin in the generated file -- if these are absent, check that the original plugin location has the appropriate files.
Now the part that needs the most actual work: create a Makefile to build your plugin code with Emscripten.
The @piper-vamp-js@ repository contains most of what you need, in the form of a file @Makefile.inc@ that you can include from your Makefile. But you need to provide all the plugin-specific source files and any additional build flags, and set the variables that @Makefile.inc@ expects. There is a comment at the top of @Makefile.inc@ that explains this further.
An example Makefile is as follows:
<pre>
PIPER_VAMP_JS_DIR := ../piper-vamp-js
PLUGIN_DIR := ../cepstral-pitchtracker
SRC_DIR := $(PLUGIN_DIR)
MODULE_NAME := CepstralPitchTracker
MODULE_SOURCE := cepstral-pitchtracker.cpp
PLUGIN_SOURCES := $(SRC_DIR)/AgentFeeder.cpp \
$(SRC_DIR)/CepstralPitchTracker.cpp \
$(SRC_DIR)/NoteHypothesis.cpp \
$(SRC_DIR)/PeakInterpolator.cpp
INCLUDES := -I$(SRC_DIR)
include $(PIPER_VAMP_JS_DIR)/Makefile.inc
</pre>
This goes into the same directory as your entry point file.
Now, to build the JS module, run
<pre>
$ make clean
$ make em
</pre>
This should produce a pair of JS files called (in this case) @CepstralPitchTracker.js@ and @CepstralPitchTracker.umd.js@. Both of these contain the same module code, just wrapped in different ways (plain Emscripten module vs UMD wrapper). Use whichever your application finds more convenient.
To test the resulting build, run
<pre>
$ make test
</pre>