Daniel@0: %DEMGLM1 Demonstrate simple classification using a generalized linear model.
Daniel@0: %
Daniel@0: %	Description
Daniel@0: %	 The problem consists of a two dimensional input matrix DATA and a
Daniel@0: %	vector of classifications T.  The data is  generated from two
Daniel@0: %	Gaussian clusters, and a generalized linear model with logistic
Daniel@0: %	output is trained using iterative reweighted least squares. A plot of
Daniel@0: %	the data together with the 0.1, 0.5 and 0.9 contour lines of the
Daniel@0: %	conditional probability is generated.
Daniel@0: %
Daniel@0: %	See also
Daniel@0: %	DEMGLM2, GLM, GLMTRAIN
Daniel@0: %
Daniel@0: 
Daniel@0: %	Copyright (c) Ian T Nabney (1996-2001)
Daniel@0: 
Daniel@0: 
Daniel@0: % Generate data from two classes in 2d
Daniel@0: input_dim = 2;
Daniel@0: 
Daniel@0: % Fix seeds for reproducible results
Daniel@0: randn('state', 42);
Daniel@0: rand('state', 42);
Daniel@0: 
Daniel@0: ndata = 100;
Daniel@0: % Generate mixture of two Gaussians in two dimensional space
Daniel@0: mix = gmm(2, 2, 'spherical');
Daniel@0: mix.priors = [0.4 0.6];              % Cluster priors 
Daniel@0: mix.centres = [2.0, 2.0; 0.0, 0.0];  % Cluster centres
Daniel@0: mix.covars = [0.5, 1.0];
Daniel@0: 
Daniel@0: [data, label] = gmmsamp(mix, ndata);
Daniel@0: targets = label - ones(ndata, 1);
Daniel@0: 
Daniel@0: % Plot the result
Daniel@0: 
Daniel@0: clc
Daniel@0: disp('This demonstration illustrates the use of a generalized linear model')
Daniel@0: disp('to classify data from two classes in a two-dimensional space. We')
Daniel@0: disp('begin by generating and plotting the data.')
Daniel@0: disp(' ')
Daniel@0: disp('Press any key to continue.')
Daniel@0: pause
Daniel@0: 
Daniel@0: fh1 = figure;
Daniel@0: plot(data(label==1,1), data(label==1,2), 'bo');
Daniel@0: hold on
Daniel@0: axis([-4 5 -4 5])
Daniel@0: set(gca, 'box', 'on')
Daniel@0: plot(data(label==2,1), data(label==2,2), 'rx')
Daniel@0: title('Data')
Daniel@0: 
Daniel@0: clc
Daniel@0: disp('Now we fit a model consisting of a logistic sigmoid function of')
Daniel@0: disp('a linear combination of the input variables.')
Daniel@0: disp(' ')
Daniel@0: disp('The model is trained using the IRLS algorithm for 5 iterations')
Daniel@0: disp(' ')
Daniel@0: disp('Press any key to continue.')
Daniel@0: pause
Daniel@0: 
Daniel@0: net = glm(input_dim, 1, 'logistic');
Daniel@0: options = foptions;
Daniel@0: options(1) = 1;
Daniel@0: options(14) = 5;
Daniel@0: net = glmtrain(net, options, data, targets);
Daniel@0: 
Daniel@0: disp(' ')
Daniel@0: disp('We now plot some density contours given by this model.')
Daniel@0: disp('The contour labelled 0.5 is the decision boundary.')
Daniel@0: disp(' ')
Daniel@0: disp('Press any key to continue.')
Daniel@0: pause
Daniel@0: x = -4.0:0.2:5.0;
Daniel@0: y = -4.0:0.2:5.0;
Daniel@0: [X, Y] = meshgrid(x,y);
Daniel@0: X = X(:);
Daniel@0: Y = Y(:);
Daniel@0: grid = [X Y];
Daniel@0: Z = glmfwd(net, grid);
Daniel@0: Z = reshape(Z, length(x), length(y));
Daniel@0: v = [0.1 0.5 0.9];
Daniel@0: [c, h] = contour(x, y, Z, v);
Daniel@0: title('Generalized Linear Model')
Daniel@0: set(h, 'linewidth', 3)
Daniel@0: clabel(c, h);
Daniel@0: 
Daniel@0: clc
Daniel@0: disp('Note that the contours of constant density are straight lines.')
Daniel@0: disp(' ')
Daniel@0: disp('Press any key to end.')
Daniel@0: pause
Daniel@0: close(fh1);
Daniel@0: clear all;
Daniel@0: