Mercurial > hg > camir-aes2014

comparison toolboxes/SVM-light/svm_light_faq.html @ 0:e9a9cd732c1e tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
first hg version after svn
author wolffd
date Tue, 10 Feb 2015 15:05:51 +0000
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:e9a9cd732c1e
1 <HTML>
2 <HEAD>
3 <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=windows-1252">
4 <META NAME="Generator" CONTENT="Microsoft FrontPage 6.0">
5 <TITLE>svm_light_faq</TITLE>
6 <META NAME="Version" CONTENT="8.0.3514">
7 <META NAME="Date" CONTENT="11/26/96">
8 <META NAME="Template" CONTENT="C:\Programme\Microsoft Office\Office\HTML.DOT">
9 </HEAD>
10 <BODY LINK="#0000ff" VLINK="#800080" BGCOLOR="#ffffff">
11
12 <TABLE CELLSPACING=0 BORDER=0 CELLPADDING=5>
13 <TR><TD WIDTH="14%" VALIGN="TOP">
14 <H2><a TARGET="_top" HREF="http://www-ai.cs.uni-dortmund.de/"><IMG SRC="eier_graybg.gif" BORDER=0 WIDTH=100 HEIGHT=81></A></H2></TD>
15 <TD WIDTH="75%" VALIGN="TOP">
16 <H1 ALIGN="CENTER">SVM<I><SUP>light</SUP> &amp;</I> SVM<sup><i>struct</i></sup><I>
17 </I> </H1>
18 <H1 ALIGN="CENTER">FAQ</H1>
19 <FONT COLOR="#000000"><P ALIGN="CENTER">Author: </FONT><a TARGET="_top" HREF="http://www.joachims.org/">Thorsten Joachims</A><FONT COLOR="#000000"> &lt;</FONT><A HREF="mailto:thorsten@joachims.org">thorsten@joachims.org</A><FONT COLOR="#000000">&gt; <BR>
20 </FONT><a TARGET="_top" HREF="http://www.cornell.edu/">Cornell University</A><FONT COLOR="#000000"> <BR>
21 </FONT><a TARGET="_top" HREF="http://www.cs.cornell.edu/">Department of Computer Science</A><FONT COLOR="#000000"> </P>
22 <P ALIGN="CENTER">Developed at: <BR>
23 </FONT><a TARGET="_top" HREF="http://www.uni-dortmund.de/">University of Dortmund</A><FONT COLOR="#000000">, </FONT><a TARGET="_top" HREF="http://www.informatik.uni-dortmund.de/">Informatik</A><FONT COLOR="#000000">, </FONT><a TARGET="_top" HREF="http://www-ai.informatik.uni-dortmund.de/">AI-Unit</A><FONT COLOR="#000000"> <BR>
24 </FONT><a TARGET="_top" HREF="http://www.sfb475.uni-dortmund.de/">Collaborative Research Center on 'Complexity Reduction in Multivariate Data' (SFB475)</A><FONT COLOR="#000000"> </FONT> </P>
25 <P ALIGN="CENTER"><FONT COLOR="#000000"> Date: 25. May, 2009</FONT></P></TD>
26 <TD WIDTH="11%" VALIGN="TOP">
27 <H2><IMG SRC="http://www.joachims.org/images/culogo_125.gif" WIDTH=80 HEIGHT=80></H2></TD>
28 </TR>
29 </TABLE>
30
31 <H2>How do I compile on a Windows PC?</H2>
32 <UL>
33 <LI>The following instructions apply to SVM<I><SUP>light</SUP></I>, SVM<sup><i>struct</i></sup>, SVM<sup><i>perf</i></sup>, SVM<sup><i>cfg</i></sup>, SVM<sup><i>multiclass</i></sup>,
34 and SVM<sup><i>hmm</i></sup>. </LI>
35 <LI>The easiest is to install <a href="http://www.cygwin.com">CYGWIN</a> and
36 use the gcc compiler that comes with the Cygwin distribution. Just open a
37 cygwin command window, change to the directory that contains file &quot;Makefile&quot; and type
38 <br>
39 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; make<br>
40 at the command prompt. This creates executables that run under Cygwin.</LI>
41 <LI>If you want execuables that do not need Cygwin to run, follow the same
42 instructions as above, but make sure that you download the MinGW package as
43 part of Cygwin. Then use the command <br>
44 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; make 'SFLAGS=-mno-cygwin'<br>
45 to create the executables.</LI>
46 <LI>You can also use the Visual C compiler, but it will be much more of a
47 hassle than using Cygwin. To use Visual C, you need to build the appropriate project file according to the provided <TT>"Makefile"</TT>.
48 Make sure you do not include the file &quot;svm_loqo.c&quot; in your project, unless you
49 want to use the PL_LOQO optimizer instead of the built-in optimizer.</p>
50 </UL>
51
52 <H2>How do I compile on a PowerMac using Code Warrior?</H2>
53
54 <UL>
55 <LI>This applies to SVM<I><SUP>light</SUP></I>, but should also work for SVM<sup><i>struct</i></sup>, SVM<sup><i>perf</i></sup>, SVM<sup><i>cfg</i></sup>, SVM<sup><i>multiclass</i></sup>,
56 and SVM<sup><i>hmm</i></sup>. </LI>
57 <LI>You need to modify the source code a little (as suggested by Jewgeni Starikow). Use <TT>#include "console.h"</TT> to emulate a UNIX shell. Then add <TT>argc=ccommand(&amp;argv)</TT> as the first instruction of each <TT>main()</TT>. </LI>
58 <LI>CPU-timing might cause another problem. The timing routines are used to calculated the runtime of the program. If you do not need this feature, remove the body of <TT>get_runtime()</TT> in <TT>svm_common.c</TT>. Otherwise replace the body with the appropriate Mac routines from <TT>'time.h'</TT>. </LI></UL>
59 <H2>How do I integrate SVM<SUP>light</SUP> into C++ code?</H2>
60
61 <UL>
62 <LI>Compile <TT>"svm_learn.c"</TT>, <TT>"svm_common.c"</TT>, and <TT>"svm_hideo.c"</TT> as C code.</LI>
63 <LI>The C++ program you want to call <TT>svm_learn/8</TT> and <TT>classify_example/2 </TT>(or <TT>classify_example_linear/2</TT>) from needs to include the following headers:<BR>
64 <FONT FACE="Courier New" SIZE=2>extern "C" {<BR>
65 # include "svm_common.h"<BR>
66 # include "svm_learn.h"<BR>
67 } </FONT> </LI>
68 <LI>Link <TT>"svm_learn.o"</TT>, <TT>"svm_common.o"</TT>, and <TT>"svm_hideo.o" </TT>to your program.</LI></UL>
69
70 <h2>Is there an option for doing multi-class classification?</h2>
71 <ul>
72 <li>Not in SVM<I><SUP>light</SUP></I>. If you do not use kernel, you can
73 use <a href="svm_multiclass.html">SVM<sup><i>multiclass</i></sup></a> to
74 do multi-class classification. Otherwise, I recommend to split the task
75 into multiple binary classification tasks for multi-class or multi-label classification. For
76 example, you can do one-class-against-the-rest classification, or pairwise classification. Refer to Chapter 2 in [<a href="index.html#References">Joachims,
77 2002a</a>].</li>
78 </ul>
79
80 <h2>Is there an option for doing cross-validation?</h2>
81 <ul>
82 <li>Yes, there is such an option in SVM<I><SUP>light</SUP></I>, but not in SVM<sup><i>struct</i></sup>.
83 By setting the option &quot;-x 1&quot;, SVM<I><SUP>light</SUP></I> computes the
84 leave-one-out estimates of the prediction error, recall, precision, and F1. By
85 default, it also outputs the XiAlpha-Estimates of these quantities. They are
86 strictly conservative approximations to the leave-one-out (i.e. they never
87 underestimate the leave-one-out error). Refer to Chapter 5 in [<a href="index.html#References">Joachims,
88 2002a</a>].</li>
89 </ul>
90 <h2>How can I output the dual variables at the solution?</h2>
91 <ul>
92 <li>Use the option -a &lt;filename&gt;. The file contains the value of the
93 dual variables (multiplied by the label y) for each training example in the
94 order in which the examples appeared in the training file.</li>
95 </ul>
96 <h2>How can I get the weight vector of the hyperplane for a linear SVM?</h2>
97 <ul>
98 <li>There is a small <a href="svm2weight.pl.txt">PERL script to
99 compute the weight vector</a> (<a href="svm2weight.py.txt">similar script in
100 PYTHON by Ori Cohen</a>) based on the model file output by svm_learn
101 and svm_perf_learn.
102 Of course, it works only in the linear case and not for kernels. All the
103 script does is compute the weighted sum of the support vectors (first
104 element in line is alpha*y, what follows is the feature vector). For
105 further info, see the comment in the model file for its format.
106 </li>
107 </ul>
108
109 <h2>What is the format of the learned model file?</h2>
110 <ul>
111 <li>The model files produced by svm_learn and svm_perf_learn are in ASCII and
112 easy to read. The first few lines contain parameters. The comments in the
113 model file are self-explanatory. The last of these is the threshold b of the
114 learned hyperplane classifier sign(w*x - b). The following lines each contain
115 a support vector, and the first element of the line is its coefficient (alpha_i
116 * y_i) in the kernel expansion. The support vectors are listed in random
117 order. Note that the term support vector has a different meaning in SVM<sup>perf</sup>
118 compared to SVM<SUP>light</SUP>.</li>
119 </ul>
120
121
122 <h2>How can I implement my own Kernel?</h2>
123 <ul>
124 <li>You can write your own kernel by extending the function &quot;double
125 custom_kernel(KERNEL_PARM *kernel_parm, SVECTOR *a, SVECTOR *b)&quot; in &quot;kernel.h&quot;
126 and then select it via the &quot;-t 4&quot; option. The &quot;a&quot; and &quot;b&quot; are pointers to the
127 examples you are computing the kernel for.</li>
128 <ul>
129 <li>If the data you are working with is vectorial, then you can use the
130 existing sparse vector operations defined in svm_common.c. For example,<br>
131 &nbsp;&nbsp;&nbsp; pow(sprod_ss(a-&gt;words,b-&gt;words),2.0) <br>
132 implements the homogeneous polynomial kernel of degree 2.</li>
133 <li>If the data is non-vectorial (e.g. strings), then you can use the
134 following functionality. In the training and test data files, whatever is
135 written behind the &quot;#&quot; is copied into the &quot;userdefined&quot; field of the
136 internal datastructure &quot;SVECTOR&quot; as a string. So, for the line<br>
137 &nbsp;&nbsp;&nbsp; +1 #abcdefg<br>
138 you will find &quot;abcdefg&quot; in &quot;svector-&gt;userdefined&quot;.</li>
139 </ul>
140 </ul>
141
142
143 <H2>Error messages and known problems:</H2>
144
145 <UL>
146 <TT><LI>ERROR: terminating optimizer - choldc failed, matrix not positive definite</TT> </LI>
147
148 <UL>
149 <LI>If the program terminates after this message, get the lastest version of PR_LOQO and SVM-Light V2.01 (or later). </LI>
150 <LI>It the program continues after this error message, don't worry :-) </LI></UL>
151
152 <I><LI>The CPU-time is negative or looks bogus:</I> </LI>
153
154 <UL>
155 <LI>To be compatible with Windows, I used the <TT>clock/0</TT> function to measure CPU-time. This function returns a long integer and counts microseconds, so that it wraps around after about 30 minutes. You could use timing routines more appropriate for your system. For example on Solaris, you might want to use the routines from <TT>'sys/times.h'</TT>.</LI></UL>
156 </UL>
157
158 <H2>The program hangs when ...</H2>
159
160 <UL>
161 <I><LI>... reading in the examples.</I> </LI>
162
163 <UL>
164 <LI>Get version 3.02 or later. </LI></UL>
165 </UL>
166
167 <H2>Convergence during learning is very slow!</H2>
168
169 <UL>
170 <I><LI>In verbose mode 2 I observe that <TT>max violation</TT> bounces around and does not really converge.</I> </LI>
171
172 <UL>
173 <LI>Use a smaller value for the option -n (default n=q). This makes sure that only n new variables enter the working set in each iteration. This can prevent zig-zagging behavior.</LI>
174 <LI>Use a smaller or larger value for the size of the working set (option -q). Reasonable values are in the range [2:50].</LI>
175 <LI>You might be using an excessively large value of C (option -c) in relation to your data. The value of C should typically be less than 1000 times 1.0/(the maximum squared Euclidian length of your feature vectors). </LI>
176 <LI>You have weired data and the convergence simply IS very slow <FONT FACE="Wingdings">&#74;</FONT>
177 . Sorry, not much you can do about it. </LI></UL>
178
179 <I><LI>Nearly all my training examples end up as support vectors.</I> </LI>
180
181 <UL>
182 <LI>Use a "stiffer" kernel (e g. a lower value of gamma for the RBF-kernel). If you pick a kernel which is very far away from the optimum, you will not get good generalization performance anyway. </LI>
183 <LI>Your data is really difficult to separate. Think about a better representation of your data. It is a bad idea - for example - if different features have values in very much different orders of magnitude. You might want to normalize all features to the range [-1,+1]. </LI></UL>
184 </UL>
185
186 <H2>It does not converge!</H2>
187
188 <UL>
189 <LI>If you are using the built-in HIDEO optimizer, get the version 3.50 or later. There used to be problems for low dimensional data sets for the old HIDEO optimizer.</LI>
190 <LI>It should always converge, unless there are numerical problems :-) </LI>
191 <LI>Numerical problems. </LI>
192
193 <UL>
194 <LI>Make sure your data is properly scaled. It is a bad idea - for example - if different features have values in different orders of magnitude. You might want to normalize all features to the range [-1,+1], especially for problems with more than 100 features. Or even better, normalize all feature vectors to Euclidian length 1.</LI></UL>
195 </UL>
196
197 <h2>It crashes!</h2>
198 <ul>
199 <li><i>It crashes during learning when using a non-linear kernel.</i>
200 <ul>
201 <li>Get version 5.00 or later. There was an initialization bug in the
202 kernel cache.</li>
203 </ul>
204 </li>
205 </ul>
206 <h2>The results look bogus!</h2>
207 <ul>
208 <li><i>When doing transductive learning, for some training set sizes it does
209 not obey the -p parameter and the results look bogus.</i>
210 <ul>
211 <li>Get version 5.00 or later. There was a bug introduced in version 4.00.</li>
212 </ul>
213 </li>
214 </ul>
215
216 <FONT COLOR="#000000">
217 <P>Last modified July 1st, 2007 by </FONT><a TARGET="_top" HREF="http://www.joachims.org">Thorsten Joachims</A><FONT COLOR="#000000"> &lt;</FONT><a HREF="mailto:thorsten@joachims.org">thorsten@joachims.org</a><FONT COLOR="#000000">&gt;</P></FONT></BODY>
218 </HTML>