Mercurial > hg > camir-aes2014

comparison toolboxes/FullBNT-1.0.7/nethelp3.3/conjgrad.htm @ 0:e9a9cd732c1e tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
first hg version after svn
author wolffd
date Tue, 10 Feb 2015 15:05:51 +0000
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:e9a9cd732c1e
1 <html>
2 <head>
3 <title>
4 Netlab Reference Manual conjgrad
5 </title>
6 </head>
7 <body>
8 <H1> conjgrad
9 </H1>
10 <h2>
11 Purpose
12 </h2>
13 Conjugate gradients optimization.
14
15 <p><h2>
16 Description
17 </h2>
18 <CODE>[x, options, flog, pointlog] = conjgrad(f, x, options, gradf)</CODE> uses a
19 conjugate gradients
20 algorithm to find the minimum of the function <CODE>f(x)</CODE> whose
21 gradient is given by <CODE>gradf(x)</CODE>. Here <CODE>x</CODE> is a row vector
22 and <CODE>f</CODE> returns a scalar value.
23 The point at which <CODE>f</CODE> has a local minimum
24 is returned as <CODE>x</CODE>. The function value at that point is returned
25 in <CODE>options(8)</CODE>. A log of the function values
26 after each cycle is (optionally) returned in <CODE>flog</CODE>, and a log
27 of the points visited is (optionally) returned in <CODE>pointlog</CODE>.
28
29 <p><CODE>conjgrad(f, x, options, gradf, p1, p2, ...)</CODE> allows
30 additional arguments to be passed to <CODE>f()</CODE> and <CODE>gradf()</CODE>.
31
32 <p>The optional parameters have the following interpretations.
33
34 <p><CODE>options(1)</CODE> is set to 1 to display error values; also logs error
35 values in the return argument <CODE>errlog</CODE>, and the points visited
36 in the return argument <CODE>pointslog</CODE>. If <CODE>options(1)</CODE> is set to 0,
37 then only warning messages are displayed. If <CODE>options(1)</CODE> is -1,
38 then nothing is displayed.
39
40 <p><CODE>options(2)</CODE> is a measure of the absolute precision required for the value
41 of <CODE>x</CODE> at the solution. If the absolute difference between
42 the values of <CODE>x</CODE> between two successive steps is less than
43 <CODE>options(2)</CODE>, then this condition is satisfied.
44
45 <p><CODE>options(3)</CODE> is a measure of the precision required of the objective
46 function at the solution. If the absolute difference between the
47 objective function values between two successive steps is less than
48 <CODE>options(3)</CODE>, then this condition is satisfied.
49 Both this and the previous condition must be
50 satisfied for termination.
51
52 <p><CODE>options(9)</CODE> is set to 1 to check the user defined gradient function.
53
54 <p><CODE>options(10)</CODE> returns the total number of function evaluations (including
55 those in any line searches).
56
57 <p><CODE>options(11)</CODE> returns the total number of gradient evaluations.
58
59 <p><CODE>options(14)</CODE> is the maximum number of iterations; default 100.
60
61 <p><CODE>options(15)</CODE> is the precision in parameter space of the line search;
62 default <CODE>1e-4</CODE>.
63
64 <p><h2>
65 Examples
66 </h2>
67 An example of
68 the use of the additional arguments is the minimization of an error
69 function for a neural network:
70 <PRE>
71
72 w = quasinew('neterr', w, options, 'netgrad', net, x, t);
73 </PRE>
74
75
76 <p><h2>
77 Algorithm
78 </h2>
79
80 The conjugate gradients algorithm constructs search
81 directions <CODE>di</CODE> that are conjugate: i.e. <CODE>di*H*d(i-1) = 0</CODE>,
82 where <CODE>H</CODE> is the Hessian matrix. This means that minimising along
83 <CODE>di</CODE> does not undo the effect of minimising along the previous
84 direction. The Polak-Ribiere formula is used to calculate new search
85 directions. The Hessian is not calculated, so there is only an
86 <CODE>O(W)</CODE> storage requirement (where <CODE>W</CODE> is the number of
87 parameters). However, relatively accurate line searches must be used
88 (default is <CODE>1e-04</CODE>).
89
90 <p><h2>
91 See Also
92 </h2>
93 <CODE><a href="graddesc.htm">graddesc</a></CODE>, <CODE><a href="linemin.htm">linemin</a></CODE>, <CODE><a href="minbrack.htm">minbrack</a></CODE>, <CODE><a href="quasinew.htm">quasinew</a></CODE>, <CODE><a href="scg.htm">scg</a></CODE><hr>
94 <b>Pages:</b>
95 <a href="index.htm">Index</a>
96 <hr>
97 <p>Copyright (c) Ian T Nabney (1996-9)
98
99
100 </body>
101 </html>