wolffd@0: function state = sample_mdp(prior, trans, act)
wolffd@0: % SAMPLE_MDP Sample a sequence of states from a Markov Decision Process.
wolffd@0: % state = sample_mdp(prior, trans, act)
wolffd@0: %
wolffd@0: % Inputs:
wolffd@0: % prior(i) = Pr(Q(1)=i)
wolffd@0: % trans{a}(i,j) = Pr(Q(t)=j | Q(t-1)=i, A(t)=a)
wolffd@0: % act(a) = A(t), so act(1) is ignored
wolffd@0: %
wolffd@0: % Output:
wolffd@0: % state is a vector of length T=length(act)
wolffd@0: 
wolffd@0: len = length(act);
wolffd@0: state = zeros(1,len);
wolffd@0: state(1) = sample_discrete(prior);
wolffd@0: for t=2:len
wolffd@0:   state(t) = sample_discrete(trans{act(t)}(state(t-1),:));
wolffd@0: end