|
Daniel@0
|
1 <html>
|
|
Daniel@0
|
2 <head>
|
|
Daniel@0
|
3 <title>
|
|
Daniel@0
|
4 Netlab Reference Manual mlp
|
|
Daniel@0
|
5 </title>
|
|
Daniel@0
|
6 </head>
|
|
Daniel@0
|
7 <body>
|
|
Daniel@0
|
8 <H1> mlp
|
|
Daniel@0
|
9 </H1>
|
|
Daniel@0
|
10 <h2>
|
|
Daniel@0
|
11 Purpose
|
|
Daniel@0
|
12 </h2>
|
|
Daniel@0
|
13 Create a 2-layer feedforward network.
|
|
Daniel@0
|
14
|
|
Daniel@0
|
15 <p><h2>
|
|
Daniel@0
|
16 Synopsis
|
|
Daniel@0
|
17 </h2>
|
|
Daniel@0
|
18 <PRE>
|
|
Daniel@0
|
19 net = mlp(nin, nhidden, nout, func)
|
|
Daniel@0
|
20 net = mlp(nin, nhidden, nout, func, prior)
|
|
Daniel@0
|
21 net = mlp(nin, nhidden, nout, func, prior, beta)
|
|
Daniel@0
|
22 </PRE>
|
|
Daniel@0
|
23
|
|
Daniel@0
|
24
|
|
Daniel@0
|
25 <p><h2>
|
|
Daniel@0
|
26 Description
|
|
Daniel@0
|
27 </h2>
|
|
Daniel@0
|
28 <CODE>net = mlp(nin, nhidden, nout, func)</CODE> takes the number of inputs,
|
|
Daniel@0
|
29 hidden units and output units for a 2-layer feed-forward network,
|
|
Daniel@0
|
30 together with a string <CODE>func</CODE> which specifies the output unit
|
|
Daniel@0
|
31 activation function, and returns a data structure <CODE>net</CODE>. The
|
|
Daniel@0
|
32 weights are drawn from a zero mean, unit variance isotropic Gaussian,
|
|
Daniel@0
|
33 with varianced scaled by the fan-in of the hidden or output units as
|
|
Daniel@0
|
34 appropriate. This makes use of the Matlab function
|
|
Daniel@0
|
35 <CODE>randn</CODE> and so the seed for the random weight initialization can be
|
|
Daniel@0
|
36 set using <CODE>randn('state', s)</CODE> where <CODE>s</CODE> is the seed value.
|
|
Daniel@0
|
37 The hidden units use the <CODE>tanh</CODE> activation function.
|
|
Daniel@0
|
38
|
|
Daniel@0
|
39 <p>The fields in <CODE>net</CODE> are
|
|
Daniel@0
|
40 <PRE>
|
|
Daniel@0
|
41
|
|
Daniel@0
|
42 type = 'mlp'
|
|
Daniel@0
|
43 nin = number of inputs
|
|
Daniel@0
|
44 nhidden = number of hidden units
|
|
Daniel@0
|
45 nout = number of outputs
|
|
Daniel@0
|
46 nwts = total number of weights and biases
|
|
Daniel@0
|
47 actfn = string describing the output unit activation function:
|
|
Daniel@0
|
48 'linear'
|
|
Daniel@0
|
49 'logistic
|
|
Daniel@0
|
50 'softmax'
|
|
Daniel@0
|
51 w1 = first-layer weight matrix
|
|
Daniel@0
|
52 b1 = first-layer bias vector
|
|
Daniel@0
|
53 w2 = second-layer weight matrix
|
|
Daniel@0
|
54 b2 = second-layer bias vector
|
|
Daniel@0
|
55 </PRE>
|
|
Daniel@0
|
56
|
|
Daniel@0
|
57 Here <CODE>w1</CODE> has dimensions <CODE>nin</CODE> times <CODE>nhidden</CODE>, <CODE>b1</CODE> has
|
|
Daniel@0
|
58 dimensions <CODE>1</CODE> times <CODE>nhidden</CODE>, <CODE>w2</CODE> has
|
|
Daniel@0
|
59 dimensions <CODE>nhidden</CODE> times <CODE>nout</CODE>, and <CODE>b2</CODE> has
|
|
Daniel@0
|
60 dimensions <CODE>1</CODE> times <CODE>nout</CODE>.
|
|
Daniel@0
|
61
|
|
Daniel@0
|
62 <p><CODE>net = mlp(nin, nhidden, nout, func, prior)</CODE>, in which <CODE>prior</CODE> is
|
|
Daniel@0
|
63 a scalar, allows the field <CODE>net.alpha</CODE> in the data structure
|
|
Daniel@0
|
64 <CODE>net</CODE> to be set, corresponding to a zero-mean isotropic Gaussian
|
|
Daniel@0
|
65 prior with inverse variance with value <CODE>prior</CODE>. Alternatively,
|
|
Daniel@0
|
66 <CODE>prior</CODE> can consist of a data structure with fields <CODE>alpha</CODE>
|
|
Daniel@0
|
67 and <CODE>index</CODE>, allowing individual Gaussian priors to be set over
|
|
Daniel@0
|
68 groups of weights in the network. Here <CODE>alpha</CODE> is a column vector
|
|
Daniel@0
|
69 in which each element corresponds to a separate group of weights,
|
|
Daniel@0
|
70 which need not be mutually exclusive. The membership of the groups is
|
|
Daniel@0
|
71 defined by the matrix <CODE>indx</CODE> in which the columns correspond to
|
|
Daniel@0
|
72 the elements of <CODE>alpha</CODE>. Each column has one element for each
|
|
Daniel@0
|
73 weight in the matrix, in the order defined by the function
|
|
Daniel@0
|
74 <CODE>mlppak</CODE>, and each element is 1 or 0 according to whether the
|
|
Daniel@0
|
75 weight is a member of the corresponding group or not. A utility
|
|
Daniel@0
|
76 function <CODE>mlpprior</CODE> is provided to help in setting up the
|
|
Daniel@0
|
77 <CODE>prior</CODE> data structure.
|
|
Daniel@0
|
78
|
|
Daniel@0
|
79 <p><CODE>net = mlp(nin, nhidden, nout, func, prior, beta)</CODE> also sets the
|
|
Daniel@0
|
80 additional field <CODE>net.beta</CODE> in the data structure <CODE>net</CODE>, where
|
|
Daniel@0
|
81 beta corresponds to the inverse noise variance.
|
|
Daniel@0
|
82
|
|
Daniel@0
|
83 <p><h2>
|
|
Daniel@0
|
84 See Also
|
|
Daniel@0
|
85 </h2>
|
|
Daniel@0
|
86 <CODE><a href="mlpprior.htm">mlpprior</a></CODE>, <CODE><a href="mlppak.htm">mlppak</a></CODE>, <CODE><a href="mlpunpak.htm">mlpunpak</a></CODE>, <CODE><a href="mlpfwd.htm">mlpfwd</a></CODE>, <CODE><a href="mlperr.htm">mlperr</a></CODE>, <CODE><a href="mlpbkp.htm">mlpbkp</a></CODE>, <CODE><a href="mlpgrad.htm">mlpgrad</a></CODE><hr>
|
|
Daniel@0
|
87 <b>Pages:</b>
|
|
Daniel@0
|
88 <a href="index.htm">Index</a>
|
|
Daniel@0
|
89 <hr>
|
|
Daniel@0
|
90 <p>Copyright (c) Ian T Nabney (1996-9)
|
|
Daniel@0
|
91
|
|
Daniel@0
|
92
|
|
Daniel@0
|
93 </body>
|
|
Daniel@0
|
94 </html> |
