|
Daniel@0
|
1 <html>
|
|
Daniel@0
|
2 <head>
|
|
Daniel@0
|
3 <title>
|
|
Daniel@0
|
4 Netlab Reference Manual conjgrad
|
|
Daniel@0
|
5 </title>
|
|
Daniel@0
|
6 </head>
|
|
Daniel@0
|
7 <body>
|
|
Daniel@0
|
8 <H1> conjgrad
|
|
Daniel@0
|
9 </H1>
|
|
Daniel@0
|
10 <h2>
|
|
Daniel@0
|
11 Purpose
|
|
Daniel@0
|
12 </h2>
|
|
Daniel@0
|
13 Conjugate gradients optimization.
|
|
Daniel@0
|
14
|
|
Daniel@0
|
15 <p><h2>
|
|
Daniel@0
|
16 Description
|
|
Daniel@0
|
17 </h2>
|
|
Daniel@0
|
18 <CODE>[x, options, flog, pointlog] = conjgrad(f, x, options, gradf)</CODE> uses a
|
|
Daniel@0
|
19 conjugate gradients
|
|
Daniel@0
|
20 algorithm to find the minimum of the function <CODE>f(x)</CODE> whose
|
|
Daniel@0
|
21 gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector
|
|
Daniel@0
|
22 and <CODE>f</CODE> returns a scalar value.
|
|
Daniel@0
|
23 The point at which <CODE>f</CODE> has a local minimum
|
|
Daniel@0
|
24 is returned as <CODE>x</CODE>. The function value at that point is returned
|
|
Daniel@0
|
25 in <CODE>options(8)</CODE>. A log of the function values
|
|
Daniel@0
|
26 after each cycle is (optionally) returned in <CODE>flog</CODE>, and a log
|
|
Daniel@0
|
27 of the points visited is (optionally) returned in <CODE>pointlog</CODE>.
|
|
Daniel@0
|
28
|
|
Daniel@0
|
29 <p><CODE>conjgrad(f, x, options, gradf, p1, p2, ...)</CODE> allows
|
|
Daniel@0
|
30 additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>.
|
|
Daniel@0
|
31
|
|
Daniel@0
|
32 <p>The optional parameters have the following interpretations.
|
|
Daniel@0
|
33
|
|
Daniel@0
|
34 <p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error
|
|
Daniel@0
|
35 values in the return argument <CODE>errlog</CODE>, and the points visited
|
|
Daniel@0
|
36 in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0,
|
|
Daniel@0
|
37 then only warning messages are displayed. If <CODE>options(1)</CODE> is -1,
|
|
Daniel@0
|
38 then nothing is displayed.
|
|
Daniel@0
|
39
|
|
Daniel@0
|
40 <p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value
|
|
Daniel@0
|
41 of <CODE>x</CODE> at the solution. If the absolute difference between
|
|
Daniel@0
|
42 the values of <CODE>x</CODE> between two successive steps is less than
|
|
Daniel@0
|
43 <CODE>options(2)</CODE>, then this condition is satisfied.
|
|
Daniel@0
|
44
|
|
Daniel@0
|
45 <p><CODE>options(3)</CODE> is a measure of the precision required of the objective
|
|
Daniel@0
|
46 function at the solution. If the absolute difference between the
|
|
Daniel@0
|
47 objective function values between two successive steps is less than
|
|
Daniel@0
|
48 <CODE>options(3)</CODE>, then this condition is satisfied.
|
|
Daniel@0
|
49 Both this and the previous condition must be
|
|
Daniel@0
|
50 satisfied for termination.
|
|
Daniel@0
|
51
|
|
Daniel@0
|
52 <p><CODE>options(9)</CODE> is set to 1 to check the user defined gradient function.
|
|
Daniel@0
|
53
|
|
Daniel@0
|
54 <p><CODE>options(10)</CODE> returns the total number of function evaluations (including
|
|
Daniel@0
|
55 those in any line searches).
|
|
Daniel@0
|
56
|
|
Daniel@0
|
57 <p><CODE>options(11)</CODE> returns the total number of gradient evaluations.
|
|
Daniel@0
|
58
|
|
Daniel@0
|
59 <p><CODE>options(14)</CODE> is the maximum number of iterations; default 100.
|
|
Daniel@0
|
60
|
|
Daniel@0
|
61 <p><CODE>options(15)</CODE> is the precision in parameter space of the line search;
|
|
Daniel@0
|
62 default <CODE>1e-4</CODE>.
|
|
Daniel@0
|
63
|
|
Daniel@0
|
64 <p><h2>
|
|
Daniel@0
|
65 Examples
|
|
Daniel@0
|
66 </h2>
|
|
Daniel@0
|
67 An example of
|
|
Daniel@0
|
68 the use of the additional arguments is the minimization of an error
|
|
Daniel@0
|
69 function for a neural network:
|
|
Daniel@0
|
70 <PRE>
|
|
Daniel@0
|
71
|
|
Daniel@0
|
72 w = quasinew('neterr', w, options, 'netgrad', net, x, t);
|
|
Daniel@0
|
73 </PRE>
|
|
Daniel@0
|
74
|
|
Daniel@0
|
75
|
|
Daniel@0
|
76 <p><h2>
|
|
Daniel@0
|
77 Algorithm
|
|
Daniel@0
|
78 </h2>
|
|
Daniel@0
|
79
|
|
Daniel@0
|
80 The conjugate gradients algorithm constructs search
|
|
Daniel@0
|
81 directions <CODE>di</CODE> that are conjugate: i.e. <CODE>di*H*d(i-1) = 0</CODE>,
|
|
Daniel@0
|
82 where <CODE>H</CODE> is the Hessian matrix. This means that minimising along
|
|
Daniel@0
|
83 <CODE>di</CODE> does not undo the effect of minimising along the previous
|
|
Daniel@0
|
84 direction. The Polak-Ribiere formula is used to calculate new search
|
|
Daniel@0
|
85 directions. The Hessian is not calculated, so there is only an
|
|
Daniel@0
|
86 <CODE>O(W)</CODE> storage requirement (where <CODE>W</CODE> is the number of
|
|
Daniel@0
|
87 parameters). However, relatively accurate line searches must be used
|
|
Daniel@0
|
88 (default is <CODE>1e-04</CODE>).
|
|
Daniel@0
|
89
|
|
Daniel@0
|
90 <p><h2>
|
|
Daniel@0
|
91 See Also
|
|
Daniel@0
|
92 </h2>
|
|
Daniel@0
|
93 <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr>
|
|
Daniel@0
|
94 <b>Pages:</b>
|
|
Daniel@0
|
95 <a href="index.htm">Index</a>
|
|
Daniel@0
|
96 <hr>
|
|
Daniel@0
|
97 <p>Copyright (c) Ian T Nabney (1996-9)
|
|
Daniel@0
|
98
|
|
Daniel@0
|
99
|
|
Daniel@0
|
100 </body>
|
|
Daniel@0
|
101 </html> |
