IDyOM Project

h1. Installing IDyOM and its prerequisites

{{>toc}}

h2. Summary

# Download "Steel Bank Common Lisp [SBCL]":http://www.sbcl.org/platform-table.html and "install it":http://www.sbcl.org/getting.html

# Install Emacs ("available here for MacOS":http://emacsformacosx.com/)

# Install "Quicklisp":http://www.quicklisp.org/beta/, a library manager for Common Lisp (download "<code>quicklisp.lisp</code>":http://beta.quicklisp.org/quicklisp.lisp and follow the instructions "here":http://www.quicklisp.org/beta/#installation)

# (Optional) Install an SQL database - I recommend  "SQLite":http://www.sqlite.org/ which is already installed on most recent flavours of MacOS and Linux. See the "CLSQL Manual":http://clsql.b9.com/manual/ for information on other database systems you can use and how to access them from Common Lisp.

# Install and setup the IDyOM code itself

The following is a detailed description of each of these steps tested on MacOS 10.7.5 and Ubuntu 14.04 (GNU/Linux 3.13.0-24-generic x86_64). It has been successfully installed on Windows following the same basic steps but I don't have a Windows machine to test the detailed instructions.

h2. 1. Install SBCL

# download SBCL for your OS and processor (e.g., for MacOS 10, choose the AMD64 version) here: http://www.sbcl.org/platform-table.html

# on MacOS make sure you have the Xcode developer tools installed from the DVD that came with your Mac (required for GNU make)

# install SBCL by following the instructions here: http://www.sbcl.org/getting.html (Basically: unpack the downloaded tar.bz2 file, open a terminal window, change directory (<code>cd</code>) into the unpacked directory and type <code>sudo sh install.sh</code>).

h2. 2. Download and install Emacs

MacOS: Universal binaries available here: http://emacsformacosx.com/

Linux: <code>sudo apt-get install emacs</code>

h2. 3. Install Quicklisp

*NB:* the full, original instructions are here http://www.quicklisp.org/beta/#installation. The following is my summary.

# make a new folder in your home directory called 'quicklisp'

# download the file http://beta.quicklisp.org/quicklisp.lisp and put it into the new folder

# open a terminal window and type 'sbcl' to start sbcl

# type each of the following lines followed by enter, changing the path to match the location of your quicklisp folder:

<pre>

(load "/Users/marcusp/quicklisp/quicklisp.lisp")

(quicklisp-quickstart:install :path "/Users/marcusp/quicklisp/")

(ql:add-to-init-file)

(ql:quickload "quicklisp-slime-helper")

</pre>

# Remember to follow the instructions given by each of these commands. *In particular*, the final command prints some lines of code to copy and paste into the emacs configuration file called <code>.emacs</code> in your home directory (you must create this file in Emacs or another text editor): 

<pre>

To use, add this to your ~/.emacs:

  (load (expand-file-name "~/quicklisp/slime-helper.el"))

  ;; Replace "sbcl" with the path to your implementation

  (setq inferior-lisp-program "sbcl")

</pre>

# open (or restart) Emacs, press Alt-x and type slime - you should now be running SBCL from within Emacs

h2. 4. Install an SQL database

MacOS: No action required, Sqlite is already installed (unless you prefer a different database system, see below).

Linux: <code>sudo apt-get install sqlite</code>

h2. 5. Install IDyOM

# download the latest release of the IDyOM code:

** "latest release: v1.1":https://code.soundsoftware.ac.uk/attachments/download/1194/idyom-1.1.zip

** "older releases":https://code.soundsoftware.ac.uk/projects/idyom-project/files 

# unzip the zip file into the folder 'local-projects' within the 'quicklisp' folder (e.g., '/Users/marcusp/quicklisp/local-projects/')

# [*Only required for releases prior to v1.1*] make a folder in your home directory called 'idyom'. Within that folder make two further folders, one called 'db', the other called 'data'. Within the 'data' folder make three further folders called 'cache', 'models' and 'resampling'.

# using Emacs or another text editor open the file called '.sbclrc' in your home directory and add the following lines to the end of the file (the file should already have some lisp code in it), changing the paths to match the location of the 'idyom' folder you created in step 3:

<pre>

;;; Load CLSQL by default

(ql:quickload "clsql")

;;; IDyOM

(defun start-idyom ()

   (defvar *idyom-root* "/Users/marcusp/idyom/")

   (ql:quickload "idyom")

   (clsql:connect '("/Users/marcusp/idyom/db/database.sqlite") :if-exists :old :database-type :sqlite3))

</pre>

# Restart Emacs, run Slime again (<code>Alt-x slime</code>) and type <code>(start-idyom)</code> at the prompt, <code>CL-USER></code>, to load IDyOM. The first time you run this, it will also download all the required third-party lisp libraries.

# The first time you run IDyOM, you must initialise the database by issuing the following command at the prompt (<code>CL-USER></code>):

<pre>

(mtp-admin:initialise-database)

</pre>

NB: if you run this command later it will delete all the contents of your database, so only run it once.

h1. Starting IDyOM

Having installed IDyOM as described above, you can start it by:

# launching Emacs

# typing <code>Alt-x slime</code>

# entering <code>(start-idyom)</code> at the Lisp prompt, <code>CL-USER></code>

h1. Troubleshooting

h2. Editing text files

If you use a text editor like Textedit (on MacOS) to edit lisp code (e.g., in the <code>.sbclrc</code> or <code>.emacs</code> files), it can insert non-ascii characters for things like curved quotation marks (smart quotes) which won't work. This is one reason (of many) to use Emacs as an editor. You never know, you might find you like it!

h2. Using SBCL within Emacs

Used within Emacs, SBCL will do pathname completion (using TAB) and expansion of "~" into your home directory and other useful things. You can also get previous commands easily.

See "here":http://common-lisp.net/project/slime/ for more on Slime, the Emacs library which achieves this (it is automatically installed by Quicklisp).

h2. The lisp debugger

If you find yourself with something like this:

<pre>

* (/ 1 0)

debugger invoked on a DIVISION-BY-ZERO:

  arithmetic error DIVISION-BY-ZERO signalled

Operation was SB-KERNEL::DIVISION, operands (1 0).

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):

  0: [ABORT] Exit debugger, returning to top level.

(SB-KERNEL::INTEGER-/-INTEGER 1 0)

0] 

</pre>

Then you have typed something that lisp doesn't like. Be sure to read the information and the restart options and enter the appropriate restart number - usually this is the one labelled <code>[ABORT]</code> but, depending on the situation, <code>[CONTINUE]</code> may also be useful.

h2. Updating or reinstalling IDyOM

To update IDyOM simply remove the previously installation from the <code>.../quicklisp/local-projects/</code> directory and unzip the new version. Then restart Emacs/SBCL/IDyOM as described above.

h2. Using different SQL database systems

You may use different kinds of databases. For example, to create an SQLite database called 'example.db' in directory <code>DIR</code>:

<pre>

(clsql:connect '("DIR/example.db") :if-exists :old :database-type :sqlite3)

</pre>

Alternatively, to connect to an existing local MySQL database:

<pre>

(clsql:connect '("localhost" "example-database" "username" "password") :if-exists :old :database-type :mysql)

</pre>

See the "CLSQL documentation":http://clsql.b9.com/manual/ for more on "connect":http://clsql.b9.com/manual/connect.html and "supported databases":http://clsql.b9.com/manual/prerequisites.html#idp8251808.

h2. Problems loading foreign database libraries:

Depending on how your system is configured, <code>clsql:connect</code> may give a "Couldn't load foreign libraries" error.  In this case, you

need to "tell CLSQL where it can find these libraries":http://clsql.b9.com/manual/appendix.html#foreignlibs, e.g.

<pre>

(clsql:push-library-path "/usr/local/mysql/lib/")

</pre>

The exact path will depend on your system.  Note that, for some database installations these libraries may not have been installed, and you should consult the database documentation.