MCMC Methods for MLP-network and Gaussian Process and Stuff – A documentation for Matlab Toolbox

MCMCstuff toolbox is a collection of Matlab functions for Bayesian inference with Markov chain Monte Carlo (MCMC) methods. This documentation introduces some of the features available in the toolbox. Introduction includes demonstrations of using Bayesian Multilayer Perceptron (MLP) network and Gaussian process in simple regression and classification problems with a hierarchical automatic relevance determination (ARD) prior for covariate related parameters. The regression problems demonstrate the use of Gaussian and Student’s t-distribution residual models and classification is demonstrated for two and three class classification problems. The use of Reversible jump Markov chain Monte Carlo (RJMCMC) method and ARD prior are demonstrated for input variable selection.

[1]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[2]  J. Geweke,et al.  Bayesian Treatment of the Independent Student- t Linear Model , 1993 .

[3]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[4]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[5]  P. Green Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[6]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[7]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[8]  Radford M. Neal Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification , 1997, physics/9701026.

[9]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[10]  Jouko Lampinen,et al.  On MCMC sampling in Bayesian MLP neural networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[11]  J. Lampinen,et al.  Bayesian Input Variable Selection Using Cross-Validation Predictive Densities and Reversible Jump MCMC , 2001 .

[12]  Jouko Lampinen,et al.  Bayesian approach for neural networks--review and case studies , 2001, Neural Networks.

[13]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[14]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[15]  Jouko Lampinen,et al.  Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities , 2002, Neural Computation.

[16]  Mark J. Schervish,et al.  Nonstationary Covariance Functions for Gaussian Process Regression , 2003, NIPS.

[17]  Radford M. Neal The Short-Cut Metropolis Method , 2005, math/0508060.

[18]  Nicholas Rose,et al.  Highly Structured Stochastic Systems , 2005, Technometrics.