# Bayes Net Toolbox for Matlab

Written by Kevin Murphy.
BNT is now available from sourceforge!

| Click to subscribe to the BNT email list ( http://groups.yahoo.com/group/BayesNetToolbox)

- Download BNT from BNT sourceforge site
- Terms and conditions of use (GNU Library GPL)
- How to use the toolbox
- Powerpoint slides on graphical models and BNT, presented to the Mathworks, June 2003
- Proposed design for gR, a graphical models toolkit in R, September 2003.
- Invited paper on BNT, published in Computing Science and Statistics, 2001.
- Other Bayes net software
- A brief introduction to Bayesian Networks
- Major features
- Supported models
- Why do I give the code away?
- Why Matlab?
- Acknowledgments

## Major features

- BNT supports many types of conditional probability distributions (nodes), and it is easy to add more.
  - Tabular (multinomial)
  - Gaussian
  - Softmax (logistic/ sigmoid)
  - Multi-layer perceptron (neural network)
  - Noisy-or
  - Deterministic

- BNT supports decision and utility nodes, as well as chance nodes, i.e., influence diagrams as well as Bayes nets.

- BNT supports static and dynamic BNs (useful for modelling dynamical systems and sequence data).

- BNT supports many different inference algorithms, and it is easy to add more.
  - Exact inference for static BNs:
    - junction tree
    - variable elimination
    - brute force enumeration (for discrete nets)
    - linear algebra (for Gaussian nets)
    - Pearl's algorithm (for polytrees)
    - quickscore (for QMR)

  - Approximate inference for static BNs:
    - likelihood weighting
    - Gibbs sampling
    - loopy belief propagation

  - Exact inference for DBNs:
    - junction tree
    - frontier algorithm
    - forwards-backwards (for HMMs)
    - Kalman-RTS (for LDSs)

  - Approximate inference for DBNs:
    - Boyen-Koller
    - factored-frontier/loopy belief propagation

- BNT supports several methods for parameter learning, and it is easy to add more.
  - Batch MLE/MAP parameter learning using EM. (Each node type has its own M method, e.g. softmax nodes use IRLS,
    and each inference engine has its own E method, so the code is fully modular.)
  - Sequential/batch Bayesian parameter learning (for fully observed tabular nodes only).

- BNT supports several methods for regularization, and it is easy to add more.
  - Any node can have its parameters clamped (made non-adjustable).
  - Any set of compatible nodes can have their parameters tied (c.f., weight sharing in a neural net).
  - Some node types (e.g., tabular) supports priors for MAP estimation.
  - Gaussian covariance matrices can be declared full or diagonal, and can be tied across states of their discrete parents (if any).

- BNT supports several methods for structure learning, and it is easy to add more.
  - Bayesian structure learning, using MCMC or local search (for fully observed tabular nodes only).
  - Constraint-based structure learning (IC/PC and IC*/FCI).

- The source code is extensively documented, object-oriented, and free, making it an excellent tool for teaching, research and rapid prototyping.

## Supported probabilistic models

It is trivial to implement all of the following probabilistic models using the toolbox.

- Static
  - Linear regression, logistic regression, hierarchical mixtures of experts
  - Naive Bayes classifiers, mixtures of Gaussians, sigmoid belief nets
  - Factor analysis, probabilistic PCA, probabilistic ICA, mixtures of these models
- Dynamic
  - HMMs, Factorial HMMs, coupled HMMs, input-output HMMs, DBNs
  - Kalman filters, ARMAX models, switching Kalman filters, tree-structured Kalman filters, multiscale AR models
- Many other combinations, for which there are (as yet) no names!

## Why do I give the code away?

- I was hoping for a Linux-style effect, whereby people would contribute their own Matlab code so that the package would grow. With a few exceptions, this has not happened, although several people have provided bug-fixes (see the acknowledgements). Perhaps the Open Bayes Project will be more succesful in this regard, although the evidence to date is not promising.

- Knowing that someone else might read your code forces one to document it properly, a good practice in any case, as anyone knows who has revisited old code. In addition, by having many "eye balls", it is easier to spot bugs.

- I believe in the concept of reproducible research. Good science requires that other people be able to replicate your experiments. Often a paper does not give enough details about how exactly an algorithm was implemented (e.g., how were the parameters chosen? what initial conditions were used?), and these can make a big difference in practice. Hence one should release the code that was actually used to generate the results in one's paper. This also prevents re-inventing the wheel.

- I was fed up with reading papers where all people do is figure out how to do exact inference and/or learning in a model which is just a trivial special case of a general Bayes net, e.g., input-output HMMs, coupled-HMMs, auto-regressive HMMs. My hope is that, by releasing general purpose software, the field can move on to more interesting questions. As Alfred North Whitehead said in 1911, "Civilization advances by extending the number of important operations that we can do without thinking about them."

## Why Matlab?

Matlab is an interactive, matrix-oriented programming language that enables one to express one's (mathematical) ideas very concisely and directly, without having to worry about annoying details like memory allocation or type checking. This considerably reduces development time and keeps code short, readable and fully portable. Matlab has excellent built-in support for many data analysis and visualization routines. In addition, there are many useful toolboxes, e.g., for neural networks, signal and image processing. The main disadvantages of Matlab are that it can be slow (which is why we are currently rewriting parts of BNT in C), and that the commercial license is expensive (although the student version is only $100 in the US).

Many people ask me why I did not use Octave, an open-source Matlab clone. The reason is that Octave does not support multi-dimensional arrays, cell arrays, objects, etc.

Click here for a more detailed comparison of matlab and other languages.

## Acknowledgments

I would like to thank numerous people for bug fixes, including: Rainer Deventer, Michael Robert James, Philippe Leray, Pedrito Maynard-Reid II, Andrew Ng, Ron Parr, Ilya Shpitser, Xuejing Sun, Ursula Sondhauss.

I would like to thank the following people for contributing code: Pierpaolo Brutti, Ali Taylan Cemgil, Tamar Kushnir, Ken Shan, Yair Weiss, Ron Zohar.

The following Intel employees have also contributed code: Qian Diao, Shan Huang, Yimin Zhang and especially Wei Hu.

I would like to thank Stuart Russell for funding me over the years as I developed BNT, and Gary Bradksi for hiring me as an intern at Intel, which has supported much of the recent developments of BNT.