wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0: svm_light_faq
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0: How do I compile on a Windows PC?
wolffd@0:
wolffd@0: - The following instructions apply to SVMlight, SVMstruct, SVMperf, SVMcfg, SVMmulticlass,
wolffd@0: and SVMhmm.
wolffd@0: - The easiest is to install CYGWIN and
wolffd@0: use the gcc compiler that comes with the Cygwin distribution. Just open a
wolffd@0: cygwin command window, change to the directory that contains file "Makefile" and type
wolffd@0:
wolffd@0: make
wolffd@0: at the command prompt. This creates executables that run under Cygwin.
wolffd@0: - If you want execuables that do not need Cygwin to run, follow the same
wolffd@0: instructions as above, but make sure that you download the MinGW package as
wolffd@0: part of Cygwin. Then use the command
wolffd@0: make 'SFLAGS=-mno-cygwin'
wolffd@0: to create the executables.
wolffd@0: - You can also use the Visual C compiler, but it will be much more of a
wolffd@0: hassle than using Cygwin. To use Visual C, you need to build the appropriate project file according to the provided "Makefile".
wolffd@0: Make sure you do not include the file "svm_loqo.c" in your project, unless you
wolffd@0: want to use the PL_LOQO optimizer instead of the built-in optimizer.
wolffd@0:
wolffd@0:
wolffd@0: How do I compile on a PowerMac using Code Warrior?
wolffd@0:
wolffd@0:
wolffd@0: - This applies to SVMlight, but should also work for SVMstruct, SVMperf, SVMcfg, SVMmulticlass,
wolffd@0: and SVMhmm.
wolffd@0: - You need to modify the source code a little (as suggested by Jewgeni Starikow). Use #include "console.h" to emulate a UNIX shell. Then add argc=ccommand(&argv) as the first instruction of each main().
wolffd@0: - CPU-timing might cause another problem. The timing routines are used to calculated the runtime of the program. If you do not need this feature, remove the body of get_runtime() in svm_common.c. Otherwise replace the body with the appropriate Mac routines from 'time.h'.
wolffd@0: How do I integrate SVMlight into C++ code?
wolffd@0:
wolffd@0:
wolffd@0: - Compile "svm_learn.c", "svm_common.c", and "svm_hideo.c" as C code.
wolffd@0: - The C++ program you want to call svm_learn/8 and classify_example/2 (or classify_example_linear/2) from needs to include the following headers:
wolffd@0: extern "C" {
wolffd@0: # include "svm_common.h"
wolffd@0: # include "svm_learn.h"
wolffd@0: }
wolffd@0: - Link "svm_learn.o", "svm_common.o", and "svm_hideo.o" to your program.
wolffd@0:
wolffd@0: Is there an option for doing multi-class classification?
wolffd@0:
wolffd@0: - Not in SVMlight. If you do not use kernel, you can
wolffd@0: use SVMmulticlass to
wolffd@0: do multi-class classification. Otherwise, I recommend to split the task
wolffd@0: into multiple binary classification tasks for multi-class or multi-label classification. For
wolffd@0: example, you can do one-class-against-the-rest classification, or pairwise classification. Refer to Chapter 2 in [Joachims,
wolffd@0: 2002a].
wolffd@0:
wolffd@0:
wolffd@0: Is there an option for doing cross-validation?
wolffd@0:
wolffd@0: - Yes, there is such an option in SVMlight, but not in SVMstruct.
wolffd@0: By setting the option "-x 1", SVMlight computes the
wolffd@0: leave-one-out estimates of the prediction error, recall, precision, and F1. By
wolffd@0: default, it also outputs the XiAlpha-Estimates of these quantities. They are
wolffd@0: strictly conservative approximations to the leave-one-out (i.e. they never
wolffd@0: underestimate the leave-one-out error). Refer to Chapter 5 in [Joachims,
wolffd@0: 2002a].
wolffd@0:
wolffd@0: How can I output the dual variables at the solution?
wolffd@0:
wolffd@0: - Use the option -a <filename>. The file contains the value of the
wolffd@0: dual variables (multiplied by the label y) for each training example in the
wolffd@0: order in which the examples appeared in the training file.
wolffd@0:
wolffd@0: How can I get the weight vector of the hyperplane for a linear SVM?
wolffd@0:
wolffd@0: - There is a small PERL script to
wolffd@0: compute the weight vector (similar script in
wolffd@0: PYTHON by Ori Cohen) based on the model file output by svm_learn
wolffd@0: and svm_perf_learn.
wolffd@0: Of course, it works only in the linear case and not for kernels. All the
wolffd@0: script does is compute the weighted sum of the support vectors (first
wolffd@0: element in line is alpha*y, what follows is the feature vector). For
wolffd@0: further info, see the comment in the model file for its format.
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0: What is the format of the learned model file?
wolffd@0:
wolffd@0: - The model files produced by svm_learn and svm_perf_learn are in ASCII and
wolffd@0: easy to read. The first few lines contain parameters. The comments in the
wolffd@0: model file are self-explanatory. The last of these is the threshold b of the
wolffd@0: learned hyperplane classifier sign(w*x - b). The following lines each contain
wolffd@0: a support vector, and the first element of the line is its coefficient (alpha_i
wolffd@0: * y_i) in the kernel expansion. The support vectors are listed in random
wolffd@0: order. Note that the term support vector has a different meaning in SVMperf
wolffd@0: compared to SVMlight.
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0: How can I implement my own Kernel?
wolffd@0:
wolffd@0: - You can write your own kernel by extending the function "double
wolffd@0: custom_kernel(KERNEL_PARM *kernel_parm, SVECTOR *a, SVECTOR *b)" in "kernel.h"
wolffd@0: and then select it via the "-t 4" option. The "a" and "b" are pointers to the
wolffd@0: examples you are computing the kernel for.
wolffd@0:
wolffd@0: - If the data you are working with is vectorial, then you can use the
wolffd@0: existing sparse vector operations defined in svm_common.c. For example,
wolffd@0: pow(sprod_ss(a->words,b->words),2.0)
wolffd@0: implements the homogeneous polynomial kernel of degree 2.
wolffd@0: - If the data is non-vectorial (e.g. strings), then you can use the
wolffd@0: following functionality. In the training and test data files, whatever is
wolffd@0: written behind the "#" is copied into the "userdefined" field of the
wolffd@0: internal datastructure "SVECTOR" as a string. So, for the line
wolffd@0: +1 #abcdefg
wolffd@0: you will find "abcdefg" in "svector->userdefined".
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0: Error messages and known problems:
wolffd@0:
wolffd@0:
wolffd@0: - ERROR: terminating optimizer - choldc failed, matrix not positive definite
wolffd@0:
wolffd@0:
wolffd@0: - If the program terminates after this message, get the lastest version of PR_LOQO and SVM-Light V2.01 (or later).
wolffd@0: - It the program continues after this error message, don't worry :-)
wolffd@0:
wolffd@0: - The CPU-time is negative or looks bogus:
wolffd@0:
wolffd@0:
wolffd@0: - To be compatible with Windows, I used the clock/0 function to measure CPU-time. This function returns a long integer and counts microseconds, so that it wraps around after about 30 minutes. You could use timing routines more appropriate for your system. For example on Solaris, you might want to use the routines from 'sys/times.h'.
wolffd@0:
wolffd@0:
wolffd@0: The program hangs when ...
wolffd@0:
wolffd@0:
wolffd@0: - ... reading in the examples.
wolffd@0:
wolffd@0:
wolffd@0: - Get version 3.02 or later.
wolffd@0:
wolffd@0:
wolffd@0: Convergence during learning is very slow!
wolffd@0:
wolffd@0:
wolffd@0: - In verbose mode 2 I observe that max violation bounces around and does not really converge.
wolffd@0:
wolffd@0:
wolffd@0: - Use a smaller value for the option -n (default n=q). This makes sure that only n new variables enter the working set in each iteration. This can prevent zig-zagging behavior.
wolffd@0: - Use a smaller or larger value for the size of the working set (option -q). Reasonable values are in the range [2:50].
wolffd@0: - You might be using an excessively large value of C (option -c) in relation to your data. The value of C should typically be less than 1000 times 1.0/(the maximum squared Euclidian length of your feature vectors).
wolffd@0: - You have weired data and the convergence simply IS very slow J
wolffd@0: . Sorry, not much you can do about it.
wolffd@0:
wolffd@0: - Nearly all my training examples end up as support vectors.
wolffd@0:
wolffd@0:
wolffd@0: - Use a "stiffer" kernel (e g. a lower value of gamma for the RBF-kernel). If you pick a kernel which is very far away from the optimum, you will not get good generalization performance anyway.
wolffd@0: - Your data is really difficult to separate. Think about a better representation of your data. It is a bad idea - for example - if different features have values in very much different orders of magnitude. You might want to normalize all features to the range [-1,+1].
wolffd@0:
wolffd@0:
wolffd@0: It does not converge!
wolffd@0:
wolffd@0:
wolffd@0: - If you are using the built-in HIDEO optimizer, get the version 3.50 or later. There used to be problems for low dimensional data sets for the old HIDEO optimizer.
wolffd@0: - It should always converge, unless there are numerical problems :-)
wolffd@0: - Numerical problems.
wolffd@0:
wolffd@0:
wolffd@0: - Make sure your data is properly scaled. It is a bad idea - for example - if different features have values in different orders of magnitude. You might want to normalize all features to the range [-1,+1], especially for problems with more than 100 features. Or even better, normalize all feature vectors to Euclidian length 1.
wolffd@0:
wolffd@0:
wolffd@0: It crashes!
wolffd@0:
wolffd@0: - It crashes during learning when using a non-linear kernel.
wolffd@0:
wolffd@0: - Get version 5.00 or later. There was an initialization bug in the
wolffd@0: kernel cache.
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0: The results look bogus!
wolffd@0:
wolffd@0: - When doing transductive learning, for some training set sizes it does
wolffd@0: not obey the -p parameter and the results look bogus.
wolffd@0:
wolffd@0: - Get version 5.00 or later. There was a bug introduced in version 4.00.
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0:
wolffd@0: Last modified July 1st, 2007 by
Thorsten Joachims <thorsten@joachims.org>
wolffd@0:
