Daniel@0: echo on;
Daniel@0: 
Daniel@0: clc;
Daniel@0: 
Daniel@0: % This is a very basic demo of the mixture of factor analyzer software
Daniel@0: % written in Matlab by	Zoubin Ghahramani
Daniel@0: %			Dept of Computer Science
Daniel@0: %			University of Toronto
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: % To demonstrate the software we generate a sample data set
Daniel@0: % from a mixture of two Gaussians
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: X1=randn(300,5);	% zero mean 5 dim Gaussian data 
Daniel@0: X2=randn(200,5)+2;	% 5 dim Gaussian data with mean [1 1 1 1 1]
Daniel@0: X=[X1;X2];		% total 500 data points from mixture
Daniel@0: 
Daniel@0: % Fitting the model is very easy. For example to fit a mixture of 2
Daniel@0: % factor analyzers with three factors each...
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: 
Daniel@0: [Lh,Ph,Mu,Pi,LL]=mfa(X,2,3);
Daniel@0: 
Daniel@0: % Lh, Ph, Mu, and Pi are the factor loadings, observervation
Daniel@0: % variances, observation means for each mixture, and mixing
Daniel@0: % proportions. LL is the vector of log likelihoods (the learning
Daniel@0: % curve). For more information type: help mfa
Daniel@0: 
Daniel@0: % to plot the learning curve (log likelihood at each step of EM)...
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: plot(LL);
Daniel@0: 
Daniel@0: % you get a more informative picture of convergence by looking at the
Daniel@0: % log of the first difference of the log likelihoods...
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: semilogy(diff(LL)); 
Daniel@0: 
Daniel@0: % you can look at some of the parameters of the fitted model... 
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: Mu
Daniel@0: 
Daniel@0: Pi
Daniel@0: 
Daniel@0: % ...to see whether they make any sense given that me know how the
Daniel@0: % data was generated. 
Daniel@0: 
Daniel@0: % you can also evaluate the log likelihood of another data set under
Daniel@0: % the model we have just fitted using the mfa_cl (for Calculate
Daniel@0: % Likelihood) function. For example, here we generate a test from the
Daniel@0: % same distribution. 
Daniel@0: 
Daniel@0: 
Daniel@0: X1=randn(300,5);
Daniel@0: X2=randn(200,5)+2;
Daniel@0: Xtest=[X1; X2];
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: mfa_cl(Xtest,Lh,Ph,Mu,Pi)
Daniel@0: 
Daniel@0: % we should expect the log likelihood of the test set to be lower than
Daniel@0: % that of the training set.
Daniel@0: 
Daniel@0: % finally, we can also fit a regular factor analyzer using the ffa
Daniel@0: % function (Fast Factor Analysis)...
Daniel@0: 
Daniel@0: pause;		% Hit any key to continue 
Daniel@0: 
Daniel@0: [L,Ph,LL]=ffa(X,3);
Daniel@0: