The Bayesian Evidence Scheme for Model Selection

The approach of the previous chapter is extended to the derivation of the model evidence as a measure for model weighting and selection. The idea is to integrate out the hyperparameters in the likelihood term by Gaussian approximation, which requires the derivation of the Hessian at the mode. The resulting expression for the model evidence is found to be an intuitively plausible generalisation of the results obtained by MacKay for Gaussian homoscedastic noise on the target. The nature of the various Ockham factors included in the evidence is discussed. The chapter concludes with a critical evaluation of the numerical inaccuracies inherent in this scheme.