|
wolffd@0
|
1 <html>
|
|
wolffd@0
|
2 <head>
|
|
wolffd@0
|
3 <title>
|
|
wolffd@0
|
4 Netlab Reference Manual glmhess
|
|
wolffd@0
|
5 </title>
|
|
wolffd@0
|
6 </head>
|
|
wolffd@0
|
7 <body>
|
|
wolffd@0
|
8 <H1> glmhess
|
|
wolffd@0
|
9 </H1>
|
|
wolffd@0
|
10 <h2>
|
|
wolffd@0
|
11 Purpose
|
|
wolffd@0
|
12 </h2>
|
|
wolffd@0
|
13 Evaluate the Hessian matrix for a generalised linear model.
|
|
wolffd@0
|
14
|
|
wolffd@0
|
15 <p><h2>
|
|
wolffd@0
|
16 Synopsis
|
|
wolffd@0
|
17 </h2>
|
|
wolffd@0
|
18 <PRE>
|
|
wolffd@0
|
19 h = glmhess(net, x, t)
|
|
wolffd@0
|
20 [h, hdata] = glmhess(net, x, t)
|
|
wolffd@0
|
21 h = glmhess(net, x, t, hdata)
|
|
wolffd@0
|
22 </PRE>
|
|
wolffd@0
|
23
|
|
wolffd@0
|
24
|
|
wolffd@0
|
25 <p><h2>
|
|
wolffd@0
|
26 Description
|
|
wolffd@0
|
27 </h2>
|
|
wolffd@0
|
28 <CODE>h = glmhess(net, x, t)</CODE> takes a GLM network data structure <CODE>net</CODE>,
|
|
wolffd@0
|
29 a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of target
|
|
wolffd@0
|
30 values and returns the full Hessian matrix <CODE>h</CODE> corresponding to
|
|
wolffd@0
|
31 the second derivatives of the negative log posterior distribution,
|
|
wolffd@0
|
32 evaluated for the current weight and bias values as defined by
|
|
wolffd@0
|
33 <CODE>net</CODE>. Note that the target data is not required in the calculation,
|
|
wolffd@0
|
34 but is included to make the interface uniform with <CODE>nethess</CODE>. For
|
|
wolffd@0
|
35 linear and logistic outputs, the computation is very simple and is
|
|
wolffd@0
|
36 done (in effect) in one line in <CODE>glmtrain</CODE>.
|
|
wolffd@0
|
37
|
|
wolffd@0
|
38 <p><CODE>[h, hdata] = glmhess(net, x, t)</CODE> returns both the Hessian matrix
|
|
wolffd@0
|
39 <CODE>h</CODE> and the contribution <CODE>hdata</CODE> arising from the data dependent
|
|
wolffd@0
|
40 term in the Hessian.
|
|
wolffd@0
|
41
|
|
wolffd@0
|
42 <p><CODE>h = glmhess(net, x, t, hdata)</CODE> takes a network data structure
|
|
wolffd@0
|
43 <CODE>net</CODE>, a matrix <CODE>x</CODE> of input values, and a matrix <CODE>t</CODE> of
|
|
wolffd@0
|
44 target values, together with the contribution <CODE>hdata</CODE> arising from
|
|
wolffd@0
|
45 the data dependent term in the Hessian, and returns the full Hessian
|
|
wolffd@0
|
46 matrix <CODE>h</CODE> corresponding to the second derivatives of the negative
|
|
wolffd@0
|
47 log posterior distribution. This version saves computation time if
|
|
wolffd@0
|
48 <CODE>hdata</CODE> has already been evaluated for the current weight and bias
|
|
wolffd@0
|
49 values.
|
|
wolffd@0
|
50
|
|
wolffd@0
|
51 <p><h2>
|
|
wolffd@0
|
52 Example
|
|
wolffd@0
|
53 </h2>
|
|
wolffd@0
|
54 The Hessian matrix is used by <CODE>glmtrain</CODE> to take a Newton step for
|
|
wolffd@0
|
55 softmax outputs.
|
|
wolffd@0
|
56 <PRE>
|
|
wolffd@0
|
57
|
|
wolffd@0
|
58 Hessian = glmhess(net, x, t);
|
|
wolffd@0
|
59 deltaw = -gradient*pinv(Hessian);
|
|
wolffd@0
|
60 </PRE>
|
|
wolffd@0
|
61
|
|
wolffd@0
|
62
|
|
wolffd@0
|
63 <p><h2>
|
|
wolffd@0
|
64 See Also
|
|
wolffd@0
|
65 </h2>
|
|
wolffd@0
|
66 <CODE><a href="glm.htm">glm</a></CODE>, <CODE><a href="glmtrain.htm">glmtrain</a></CODE>, <CODE><a href="hesschek.htm">hesschek</a></CODE>, <CODE><a href="nethess.htm">nethess</a></CODE><hr>
|
|
wolffd@0
|
67 <b>Pages:</b>
|
|
wolffd@0
|
68 <a href="index.htm">Index</a>
|
|
wolffd@0
|
69 <hr>
|
|
wolffd@0
|
70 <p>Copyright (c) Ian T Nabney (1996-9)
|
|
wolffd@0
|
71
|
|
wolffd@0
|
72
|
|
wolffd@0
|
73 </body>
|
|
wolffd@0
|
74 </html> |
