|
wolffd@0
|
1 function [h, hdata] = mlphess(net, x, t, hdata)
|
|
wolffd@0
|
2 %MLPHESS Evaluate the Hessian matrix for a multi-layer perceptron network.
|
|
wolffd@0
|
3 %
|
|
wolffd@0
|
4 % Description
|
|
wolffd@0
|
5 % H = MLPHESS(NET, X, T) takes an MLP network data structure NET, a
|
|
wolffd@0
|
6 % matrix X of input values, and a matrix T of target values and returns
|
|
wolffd@0
|
7 % the full Hessian matrix H corresponding to the second derivatives of
|
|
wolffd@0
|
8 % the negative log posterior distribution, evaluated for the current
|
|
wolffd@0
|
9 % weight and bias values as defined by NET.
|
|
wolffd@0
|
10 %
|
|
wolffd@0
|
11 % [H, HDATA] = MLPHESS(NET, X, T) returns both the Hessian matrix H and
|
|
wolffd@0
|
12 % the contribution HDATA arising from the data dependent term in the
|
|
wolffd@0
|
13 % Hessian.
|
|
wolffd@0
|
14 %
|
|
wolffd@0
|
15 % H = MLPHESS(NET, X, T, HDATA) takes a network data structure NET, a
|
|
wolffd@0
|
16 % matrix X of input values, and a matrix T of target values, together
|
|
wolffd@0
|
17 % with the contribution HDATA arising from the data dependent term in
|
|
wolffd@0
|
18 % the Hessian, and returns the full Hessian matrix H corresponding to
|
|
wolffd@0
|
19 % the second derivatives of the negative log posterior distribution.
|
|
wolffd@0
|
20 % This version saves computation time if HDATA has already been
|
|
wolffd@0
|
21 % evaluated for the current weight and bias values.
|
|
wolffd@0
|
22 %
|
|
wolffd@0
|
23 % See also
|
|
wolffd@0
|
24 % MLP, HESSCHEK, MLPHDOTV, EVIDENCE
|
|
wolffd@0
|
25 %
|
|
wolffd@0
|
26
|
|
wolffd@0
|
27 % Copyright (c) Ian T Nabney (1996-2001)
|
|
wolffd@0
|
28
|
|
wolffd@0
|
29 % Check arguments for consistency
|
|
wolffd@0
|
30 errstring = consist(net, 'mlp', x, t);
|
|
wolffd@0
|
31 if ~isempty(errstring);
|
|
wolffd@0
|
32 error(errstring);
|
|
wolffd@0
|
33 end
|
|
wolffd@0
|
34
|
|
wolffd@0
|
35 if nargin == 3
|
|
wolffd@0
|
36 % Data term in Hessian needs to be computed
|
|
wolffd@0
|
37 hdata = datahess(net, x, t);
|
|
wolffd@0
|
38 end
|
|
wolffd@0
|
39
|
|
wolffd@0
|
40 [h, hdata] = hbayes(net, hdata);
|
|
wolffd@0
|
41
|
|
wolffd@0
|
42 % Sub-function to compute data part of Hessian
|
|
wolffd@0
|
43 function hdata = datahess(net, x, t)
|
|
wolffd@0
|
44
|
|
wolffd@0
|
45 hdata = zeros(net.nwts, net.nwts);
|
|
wolffd@0
|
46
|
|
wolffd@0
|
47 for v = eye(net.nwts);
|
|
wolffd@0
|
48 hdata(find(v),:) = mlphdotv(net, x, t, v);
|
|
wolffd@0
|
49 end
|
|
wolffd@0
|
50
|
|
wolffd@0
|
51 return
|
