|
Daniel@0
|
1 <html>
|
|
Daniel@0
|
2 <head>
|
|
Daniel@0
|
3 <title>
|
|
Daniel@0
|
4 Netlab Reference Manual scg
|
|
Daniel@0
|
5 </title>
|
|
Daniel@0
|
6 </head>
|
|
Daniel@0
|
7 <body>
|
|
Daniel@0
|
8 <H1> scg
|
|
Daniel@0
|
9 </H1>
|
|
Daniel@0
|
10 <h2>
|
|
Daniel@0
|
11 Purpose
|
|
Daniel@0
|
12 </h2>
|
|
Daniel@0
|
13 Scaled conjugate gradient optimization.
|
|
Daniel@0
|
14
|
|
Daniel@0
|
15 <p><h2>
|
|
Daniel@0
|
16 Description
|
|
Daniel@0
|
17 </h2>
|
|
Daniel@0
|
18 <CODE>[x, options] = scg(f, x, options, gradf)</CODE> uses a scaled conjugate
|
|
Daniel@0
|
19 gradients
|
|
Daniel@0
|
20 algorithm to find a local minimum of the function <CODE>f(x)</CODE> whose
|
|
Daniel@0
|
21 gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector
|
|
Daniel@0
|
22 and <CODE>f</CODE> returns a scalar value.
|
|
Daniel@0
|
23 The point at which <CODE>f</CODE> has a local minimum
|
|
Daniel@0
|
24 is returned as <CODE>x</CODE>. The function value at that point is returned
|
|
Daniel@0
|
25 in <CODE>options(8)</CODE>.
|
|
Daniel@0
|
26
|
|
Daniel@0
|
27 <p><CODE>[x, options, flog, pointlog, scalelog] = scg(f, x, options, gradf)</CODE>
|
|
Daniel@0
|
28 also returns (optionally) a log of the function values
|
|
Daniel@0
|
29 after each cycle in <CODE>flog</CODE>, a log
|
|
Daniel@0
|
30 of the points visited in <CODE>pointlog</CODE>, and a log of the scale values
|
|
Daniel@0
|
31 in the algorithm in <CODE>scalelog</CODE>.
|
|
Daniel@0
|
32
|
|
Daniel@0
|
33 <p><CODE>scg(f, x, options, gradf, p1, p2, ...)</CODE> allows
|
|
Daniel@0
|
34 additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>.
|
|
Daniel@0
|
35
|
|
Daniel@0
|
36 The optional parameters have the following interpretations.
|
|
Daniel@0
|
37
|
|
Daniel@0
|
38 <p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error
|
|
Daniel@0
|
39 values in the return argument <CODE>errlog</CODE>, and the points visited
|
|
Daniel@0
|
40 in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0,
|
|
Daniel@0
|
41 then only warning messages are displayed. If <CODE>options(1)</CODE> is -1,
|
|
Daniel@0
|
42 then nothing is displayed.
|
|
Daniel@0
|
43
|
|
Daniel@0
|
44 <p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value
|
|
Daniel@0
|
45 of <CODE>x</CODE> at the solution. If the absolute difference between
|
|
Daniel@0
|
46 the values of <CODE>x</CODE> between two successive steps is less than
|
|
Daniel@0
|
47 <CODE>options(2)</CODE>, then this condition is satisfied.
|
|
Daniel@0
|
48
|
|
Daniel@0
|
49 <p><CODE>options(3)</CODE> is a measure of the precision required of the objective
|
|
Daniel@0
|
50 function at the solution. If the absolute difference between the
|
|
Daniel@0
|
51 objective function values between two successive steps is less than
|
|
Daniel@0
|
52 <CODE>options(3)</CODE>, then this condition is satisfied.
|
|
Daniel@0
|
53 Both this and the previous condition must be
|
|
Daniel@0
|
54 satisfied for termination.
|
|
Daniel@0
|
55
|
|
Daniel@0
|
56 <p><CODE>options(9)</CODE> is set to 1 to check the user defined gradient function.
|
|
Daniel@0
|
57
|
|
Daniel@0
|
58 <p><CODE>options(10)</CODE> returns the total number of function evaluations (including
|
|
Daniel@0
|
59 those in any line searches).
|
|
Daniel@0
|
60
|
|
Daniel@0
|
61 <p><CODE>options(11)</CODE> returns the total number of gradient evaluations.
|
|
Daniel@0
|
62
|
|
Daniel@0
|
63 <p><CODE>options(14)</CODE> is the maximum number of iterations; default 100.
|
|
Daniel@0
|
64
|
|
Daniel@0
|
65 <p><h2>
|
|
Daniel@0
|
66 Examples
|
|
Daniel@0
|
67 </h2>
|
|
Daniel@0
|
68 An example of
|
|
Daniel@0
|
69 the use of the additional arguments is the minimization of an error
|
|
Daniel@0
|
70 function for a neural network:
|
|
Daniel@0
|
71 <PRE>
|
|
Daniel@0
|
72
|
|
Daniel@0
|
73 w = scg('neterr', w, options, 'netgrad', net, x, t);
|
|
Daniel@0
|
74 </PRE>
|
|
Daniel@0
|
75
|
|
Daniel@0
|
76
|
|
Daniel@0
|
77 <p><h2>
|
|
Daniel@0
|
78 Algorithm
|
|
Daniel@0
|
79 </h2>
|
|
Daniel@0
|
80 The search direction is re-started after every <CODE>nparams</CODE>
|
|
Daniel@0
|
81 successful weight updates where <CODE>nparams</CODE> is the total number of
|
|
Daniel@0
|
82 parameters in <CODE>x</CODE>. The algorithm is based on that given by Williams
|
|
Daniel@0
|
83 (1991), with a simplified procedure for updating <CODE>lambda</CODE> when
|
|
Daniel@0
|
84 <CODE>rho < 0.25</CODE>.
|
|
Daniel@0
|
85
|
|
Daniel@0
|
86 <p><h2>
|
|
Daniel@0
|
87 See Also
|
|
Daniel@0
|
88 </h2>
|
|
Daniel@0
|
89 <CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE><hr>
|
|
Daniel@0
|
90 <b>Pages:</b>
|
|
Daniel@0
|
91 <a href="index.htm">Index</a>
|
|
Daniel@0
|
92 <hr>
|
|
Daniel@0
|
93 <p>Copyright (c) Ian T Nabney (1996-9)
|
|
Daniel@0
|
94
|
|
Daniel@0
|
95
|
|
Daniel@0
|
96 </body>
|
|
Daniel@0
|
97 </html> |
