|
Daniel@0
|
1 <html>
|
|
Daniel@0
|
2 <head>
|
|
Daniel@0
|
3 <title>
|
|
Daniel@0
|
4 Netlab Reference Manual quasinew
|
|
Daniel@0
|
5 </title>
|
|
Daniel@0
|
6 </head>
|
|
Daniel@0
|
7 <body>
|
|
Daniel@0
|
8 <H1> quasinew
|
|
Daniel@0
|
9 </H1>
|
|
Daniel@0
|
10 <h2>
|
|
Daniel@0
|
11 Purpose
|
|
Daniel@0
|
12 </h2>
|
|
Daniel@0
|
13 Quasi-Newton optimization.
|
|
Daniel@0
|
14
|
|
Daniel@0
|
15 <p><h2>
|
|
Daniel@0
|
16 Description
|
|
Daniel@0
|
17 </h2>
|
|
Daniel@0
|
18 <CODE>[x, options, flog, pointlog] = quasinew(f, x, options, gradf)</CODE>
|
|
Daniel@0
|
19 uses a quasi-Newton
|
|
Daniel@0
|
20 algorithm to find a local minimum of the function <CODE>f(x)</CODE> whose
|
|
Daniel@0
|
21 gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector
|
|
Daniel@0
|
22 and <CODE>f</CODE> returns a scalar value.
|
|
Daniel@0
|
23 The point at which <CODE>f</CODE> has a local minimum
|
|
Daniel@0
|
24 is returned as <CODE>x</CODE>. The function value at that point is returned
|
|
Daniel@0
|
25 in <CODE>options(8)</CODE>. A log of the function values
|
|
Daniel@0
|
26 after each cycle is (optionally) returned in <CODE>flog</CODE>, and a log
|
|
Daniel@0
|
27 of the points visited is (optionally) returned in <CODE>pointlog</CODE>.
|
|
Daniel@0
|
28
|
|
Daniel@0
|
29 <p><CODE>quasinew(f, x, options, gradf, p1, p2, ...)</CODE> allows
|
|
Daniel@0
|
30 additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>.
|
|
Daniel@0
|
31
|
|
Daniel@0
|
32 <p>The optional parameters have the following interpretations.
|
|
Daniel@0
|
33
|
|
Daniel@0
|
34 <p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error
|
|
Daniel@0
|
35 values in the return argument <CODE>errlog</CODE>, and the points visited
|
|
Daniel@0
|
36 in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0,
|
|
Daniel@0
|
37 then only warning messages are displayed. If <CODE>options(1)</CODE> is -1,
|
|
Daniel@0
|
38 then nothing is displayed.
|
|
Daniel@0
|
39
|
|
Daniel@0
|
40 <p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value
|
|
Daniel@0
|
41 of <CODE>x</CODE> at the solution. If the absolute difference between
|
|
Daniel@0
|
42 the values of <CODE>x</CODE> between two successive steps is less than
|
|
Daniel@0
|
43 <CODE>options(2)</CODE>, then this condition is satisfied.
|
|
Daniel@0
|
44
|
|
Daniel@0
|
45 <p><CODE>options(3)</CODE> is a measure of the precision required of the objective
|
|
Daniel@0
|
46 function at the solution. If the absolute difference between the
|
|
Daniel@0
|
47 objective function values between two successive steps is less than
|
|
Daniel@0
|
48 <CODE>options(3)</CODE>, then this condition is satisfied.
|
|
Daniel@0
|
49 Both this and the previous condition must be
|
|
Daniel@0
|
50 satisfied for termination.
|
|
Daniel@0
|
51
|
|
Daniel@0
|
52 <p><CODE>options(9)</CODE> should be set to 1 to check the user defined gradient
|
|
Daniel@0
|
53 function.
|
|
Daniel@0
|
54
|
|
Daniel@0
|
55 <p><CODE>options(10)</CODE> returns the total number of function evaluations (including
|
|
Daniel@0
|
56 those in any line searches).
|
|
Daniel@0
|
57
|
|
Daniel@0
|
58 <p><CODE>options(11)</CODE> returns the total number of gradient evaluations.
|
|
Daniel@0
|
59
|
|
Daniel@0
|
60 <p><CODE>options(14)</CODE> is the maximum number of iterations; default 100.
|
|
Daniel@0
|
61
|
|
Daniel@0
|
62 <p><CODE>options(15)</CODE> is the precision in parameter space of the line search;
|
|
Daniel@0
|
63 default <CODE>1e-2</CODE>.
|
|
Daniel@0
|
64
|
|
Daniel@0
|
65 <p><h2>
|
|
Daniel@0
|
66 Examples
|
|
Daniel@0
|
67 </h2>
|
|
Daniel@0
|
68 An example of
|
|
Daniel@0
|
69 the use of the additional arguments is the minimization of an error
|
|
Daniel@0
|
70 function for a neural network:
|
|
Daniel@0
|
71 <PRE>
|
|
Daniel@0
|
72
|
|
Daniel@0
|
73 w = quasinew('neterr', w, options, 'netgrad', net, x, t);
|
|
Daniel@0
|
74 </PRE>
|
|
Daniel@0
|
75
|
|
Daniel@0
|
76
|
|
Daniel@0
|
77 <p><h2>
|
|
Daniel@0
|
78 Algorithm
|
|
Daniel@0
|
79 </h2>
|
|
Daniel@0
|
80
|
|
Daniel@0
|
81 The quasi-Newton algorithm builds up an
|
|
Daniel@0
|
82 approximation to the inverse Hessian over a number of steps. The
|
|
Daniel@0
|
83 method requires order W squared storage, where W is the number of function
|
|
Daniel@0
|
84 parameters. The Broyden-Fletcher-Goldfarb-Shanno formula for the
|
|
Daniel@0
|
85 inverse Hessian updates is used. The line searches are carried out to
|
|
Daniel@0
|
86 a relatively low precision (1.0e-2).
|
|
Daniel@0
|
87
|
|
Daniel@0
|
88 <p><h2>
|
|
Daniel@0
|
89 See Also
|
|
Daniel@0
|
90 </h2>
|
|
Daniel@0
|
91 <CODE><a href="conjgrad.htm">conjgrad</a></CODE>, <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr>
|
|
Daniel@0
|
92 <b>Pages:</b>
|
|
Daniel@0
|
93 <a href="index.htm">Index</a>
|
|
Daniel@0
|
94 <hr>
|
|
Daniel@0
|
95 <p>Copyright (c) Ian T Nabney (1996-9)
|
|
Daniel@0
|
96
|
|
Daniel@0
|
97
|
|
Daniel@0
|
98 </body>
|
|
Daniel@0
|
99 </html> |
