论文信息 - Bayesian training of mixture density networks

Bayesian training of mixture density networks

Mixture Density Networks (MDNs) are a well-established method for modelling the conditional probability density which is useful for complex multi-valued functions where regression methods (such as MLPs) fail. In this paper we extend earlier research of a regularisation method for a special case of MDNs to the general case using evidence based regularisation and we show how the Hessian of the MDN error function can be evaluated using R-propagation. The method is tested on two data sets and compared with early stopping.

Ian T. Nabney | Lars U. Hjorth | I. Nabney | Lars U. Hjorth

[1] David J. C. MacKay,et al. A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[2] John Scott Bridle,et al. Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition , 1989, NATO Neurocomputing.

[3] Christopher M. Bishop,et al. Regression with Input-Dependent Noise: A Bayesian Treatment , 1996, NIPS.

[4] Geoffrey E. Hinton,et al. Bayesian Learning for Neural Networks , 1995 .

[5] Dirk Husmeier,et al. Modelling Conditional Probability Densities with Neural Networks , 1997 .

[6] C. Bishop. Mixture density networks , 1994 .

[7] A. Weigend,et al. Estimating the mean and variance of the target probability distribution , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[8] P. M. Williams,et al. Using Neural Networks to Model Conditional Multivariate Densities , 1996, Neural Computation.

[9] Barak A. Pearlmutter. Fast Exact Multiplication by the Hessian , 1994, Neural Computation.

[10] David J. C. MacKay,et al. Bayesian Interpolation , 1992, Neural Computation.

[11] Ian T. Nabney,et al. Regularisation of mixture density networks , 1999 .