Daniel@0: function net = mdn(nin, nhidden, ncentres, dim_target, mix_type, ...
Daniel@0: 	prior, beta)
Daniel@0: %MDN	Creates a Mixture Density Network with specified architecture.
Daniel@0: %
Daniel@0: %	Description
Daniel@0: %	NET = MDN(NIN, NHIDDEN, NCENTRES, DIMTARGET) takes the number of
Daniel@0: %	inputs,  hidden units for a 2-layer feed-forward  network and the
Daniel@0: %	number of centres and target dimension for the  mixture model whose
Daniel@0: %	parameters are set from the outputs of the neural network. The fifth
Daniel@0: %	argument MIXTYPE is used to define the type of mixture model.
Daniel@0: %	(Currently there is only one type supported: a mixture of Gaussians
Daniel@0: %	with a single covariance parameter for each component.) For this
Daniel@0: %	model, the mixture coefficients are computed from a group of softmax
Daniel@0: %	outputs, the centres are equal to a group of linear outputs, and the
Daniel@0: %	variances are  obtained by applying the exponential function to a
Daniel@0: %	third group of outputs.
Daniel@0: %
Daniel@0: %	The network is initialised by a call to MLP, and the arguments PRIOR,
Daniel@0: %	and BETA have the same role as for that function. Weight
Daniel@0: %	initialisation uses the Matlab function RANDN  and so the seed for
Daniel@0: %	the random weight initialization can be  set using RANDN('STATE', S)
Daniel@0: %	where S is the seed value. A specialised data structure (rather than
Daniel@0: %	GMM) is used for the mixture model outputs to improve the efficiency
Daniel@0: %	of error and gradient calculations in network training. The fields
Daniel@0: %	are described in MDNFWD where they are set up.
Daniel@0: %
Daniel@0: %	The fields in NET are
Daniel@0: %	  
Daniel@0: %	  type = 'mdn'
Daniel@0: %	  nin = number of input variables
Daniel@0: %	  nout = dimension of target space (not number of network outputs)
Daniel@0: %	  nwts = total number of weights and biases
Daniel@0: %	  mdnmixes = data structure for mixture model output
Daniel@0: %	  mlp = data structure for MLP network
Daniel@0: %
Daniel@0: %	See also
Daniel@0: %	MDNFWD, MDNERR, MDN2GMM, MDNGRAD, MDNPAK, MDNUNPAK, MLP
Daniel@0: %
Daniel@0: 
Daniel@0: %	Copyright (c) Ian T Nabney (1996-2001)
Daniel@0: %	David J Evans (1998)
Daniel@0: 
Daniel@0: % Currently ignore type argument: reserved for future use
Daniel@0: net.type = 'mdn';
Daniel@0: 
Daniel@0: % Set up the mixture model part of the structure
Daniel@0: % For efficiency we use a specialised data structure in place of GMM
Daniel@0: mdnmixes.type = 'mdnmixes';
Daniel@0: mdnmixes.ncentres = ncentres;
Daniel@0: mdnmixes.dim_target = dim_target;
Daniel@0: 
Daniel@0: % This calculation depends on spherical variances
Daniel@0: mdnmixes.nparams = ncentres + ncentres*dim_target + ncentres;
Daniel@0: 
Daniel@0: % Make the weights in the mdnmixes structure null 
Daniel@0: mdnmixes.mixcoeffs = [];
Daniel@0: mdnmixes.centres = [];
Daniel@0: mdnmixes.covars = [];
Daniel@0: 
Daniel@0: % Number of output nodes = number of parameters in mixture model
Daniel@0: nout = mdnmixes.nparams;
Daniel@0: 
Daniel@0: % Set up the MLP part of the network
Daniel@0: if (nargin == 5)
Daniel@0:   mlpnet = mlp(nin, nhidden, nout, 'linear');
Daniel@0: elseif (nargin == 6)
Daniel@0:   mlpnet = mlp(nin, nhidden, nout, 'linear', prior);
Daniel@0: elseif (nargin == 7)
Daniel@0:   mlpnet = mlp(nin, nhidden, nout, 'linear', prior, beta);
Daniel@0: end
Daniel@0: 
Daniel@0: % Create descriptor
Daniel@0: net.mdnmixes = mdnmixes;
Daniel@0: net.mlp = mlpnet;
Daniel@0: net.nin = nin;
Daniel@0: net.nout = dim_target;
Daniel@0: net.nwts = mlpnet.nwts;