|
wolffd@0
|
1 <html>
|
|
wolffd@0
|
2 <head>
|
|
wolffd@0
|
3 <title>
|
|
wolffd@0
|
4 Netlab Reference Manual mlphess
|
|
wolffd@0
|
5 </title>
|
|
wolffd@0
|
6 </head>
|
|
wolffd@0
|
7 <body>
|
|
wolffd@0
|
8 <H1> mlphess
|
|
wolffd@0
|
9 </H1>
|
|
wolffd@0
|
10 <h2>
|
|
wolffd@0
|
11 Purpose
|
|
wolffd@0
|
12 </h2>
|
|
wolffd@0
|
13 Evaluate the Hessian matrix for a multi-layer perceptron network.
|
|
wolffd@0
|
14
|
|
wolffd@0
|
15 <p><h2>
|
|
wolffd@0
|
16 Synopsis
|
|
wolffd@0
|
17 </h2>
|
|
wolffd@0
|
18 <PRE>
|
|
wolffd@0
|
19 h = mlphess(net, x, t)
|
|
wolffd@0
|
20 [h, hdata] = mlphess(net, x, t)
|
|
wolffd@0
|
21 h = mlphess(net, x, t, hdata)
|
|
wolffd@0
|
22 </PRE>
|
|
wolffd@0
|
23
|
|
wolffd@0
|
24
|
|
wolffd@0
|
25 <p><h2>
|
|
wolffd@0
|
26 Description
|
|
wolffd@0
|
27 </h2>
|
|
wolffd@0
|
28 <CODE>h = mlphess(net, x, t)</CODE> takes an MLP network data structure <CODE>net</CODE>,
|
|
wolffd@0
|
29 a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of target
|
|
wolffd@0
|
30 values and returns the full Hessian matrix <CODE>h</CODE> corresponding to
|
|
wolffd@0
|
31 the second derivatives of the negative log posterior distribution,
|
|
wolffd@0
|
32 evaluated for the current weight and bias values as defined by
|
|
wolffd@0
|
33 <CODE>net</CODE>.
|
|
wolffd@0
|
34
|
|
wolffd@0
|
35 <p><CODE>[h, hdata] = mlphess(net, x, t)</CODE> returns both the Hessian matrix
|
|
wolffd@0
|
36 <CODE>h</CODE> and the contribution <CODE>hdata</CODE> arising from the data dependent
|
|
wolffd@0
|
37 term in the Hessian.
|
|
wolffd@0
|
38
|
|
wolffd@0
|
39 <p><CODE>h = mlphess(net, x, t, hdata)</CODE> takes a network data structure
|
|
wolffd@0
|
40 <CODE>net</CODE>, a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of
|
|
wolffd@0
|
41 target values, together with the contribution <CODE>hdata</CODE> arising from
|
|
wolffd@0
|
42 the data dependent term in the Hessian, and returns the full Hessian
|
|
wolffd@0
|
43 matrix <CODE>h</CODE> corresponding to the second derivatives of the negative
|
|
wolffd@0
|
44 log posterior distribution. This version saves computation time if
|
|
wolffd@0
|
45 <CODE>hdata</CODE> has already been evaluated for the current weight and bias
|
|
wolffd@0
|
46 values.
|
|
wolffd@0
|
47
|
|
wolffd@0
|
48 <p><h2>
|
|
wolffd@0
|
49 Example
|
|
wolffd@0
|
50 </h2>
|
|
wolffd@0
|
51 For the standard regression framework with a Gaussian conditional
|
|
wolffd@0
|
52 distribution of target values given input values, and a simple
|
|
wolffd@0
|
53 Gaussian prior over weights, the Hessian takes the form
|
|
wolffd@0
|
54 <PRE>
|
|
wolffd@0
|
55
|
|
wolffd@0
|
56 h = beta*hd + alpha*I
|
|
wolffd@0
|
57 </PRE>
|
|
wolffd@0
|
58
|
|
wolffd@0
|
59 where the contribution <CODE>hd</CODE> is evaluated by calls to <CODE>mlphdotv</CODE> and
|
|
wolffd@0
|
60 <CODE>h</CODE> is the full Hessian.
|
|
wolffd@0
|
61
|
|
wolffd@0
|
62 <p><h2>
|
|
wolffd@0
|
63 See Also
|
|
wolffd@0
|
64 </h2>
|
|
wolffd@0
|
65 <CODE><a href="mlp.htm">mlp</a></CODE>, <CODE><a href="hesschek.htm">hesschek</a></CODE>, <CODE><a href="mlphdotv.htm">mlphdotv</a></CODE>, <CODE><a href="evidence.htm">evidence</a></CODE><hr>
|
|
wolffd@0
|
66 <b>Pages:</b>
|
|
wolffd@0
|
67 <a href="index.htm">Index</a>
|
|
wolffd@0
|
68 <hr>
|
|
wolffd@0
|
69 <p>Copyright (c) Ian T Nabney (1996-9)
|
|
wolffd@0
|
70
|
|
wolffd@0
|
71
|
|
wolffd@0
|
72 </body>
|
|
wolffd@0
|
73 </html> |
