Approximate maximum likelihood hyperparameter estimation for Gibbs priors

We describe an approximate ML estimator for the hyperparameters of a Gibbs prior which can be computed simultaneously with a maximum a posteriori (MAP) image estimate. The algorithm is based on a mean field approximation technique through which multidimensional Gibbs distributions are approximated by a separable function equal to a product of one dimensional densities. We show how this approach can be used to simplify the ML estimation problem. We also show how the Gibbs-Bogoliubov-Feynman bound can be used to optimize the approximation for a restricted class of problems.

[1]  J. Powell.  Mathematical Methods in Physics , 1965 .

[2]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[3]  G. Wahba.  Smoothing noisy data with spline functions , 1975 .

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[6]  J. Varah.  Pitfalls in the Numerical Solution of Linear Ill-Posed Problems , 1981 .

[7]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  José L. Marroquín,et al.  Probabilistic solution of inverse problems , 1985 .

[9]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[10]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[11]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[12]  Stuart Geman,et al.  Statistical methods for tomographic image reconstruction , 1987 .

[13]  D. Chandler,et al.  Introduction To Modern Statistical Mechanics , 1987 .

[14]  Sridhar Lakshmanan,et al.  Simultaneous Parameter Estimation and Segmentation of Gibbs Random Fields Using Simulated Annealing , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  J. Waldvogel,et al.  Numerical Analysis: A Comprehensive Introduction , 1989 .

[16]  G. Wahba.  Spline models for observational data , 1990 .

[17]  Federico Girosi,et al.  Parallel and deterministic algorithms from MRFs: surface reconstruction and integration , 1990, ECCV.

[18]  D. M. Titterington,et al.  A Study of Methods of Choosing the Smoothing Parameter in Image Restoration by Regularization , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Richard M. Leahy,et al.  Incorporation of Anatomical MR Data for Improved Dunctional Imaging with PET , 1991, IPMI.

[20]  Chin-Tu Chen,et al.  Image Restoration Using Gibbs Priors: Boundary Modeling, Treatment of Blurring, and Selection of Hyperparameter , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Per Christian Hansen,et al.  Analysis of Discrete Ill-Posed Problems by Means of the L-Curve , 1992, SIAM Rev..

[22]  Richard M. Leahy,et al.  Statistic-based MAP image-reconstruction from Poisson data using Gibbs priors , 1992, IEEE Trans. Signal Process..

[23]  Anand Rangarajan,et al.  A continuation method for emission tomography , 1992 .

[24]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[25]  Jun Zhang.  The mean field theory in EM procedures for Markov random fields , 1992, IEEE Trans. Signal Process..

[26]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[27]  Jun Zhang,et al.  The mean field theory in EM procedures for blind Markov random field image restoration , 1993, IEEE Trans. Image Process..

[28]  J. Zhang,et al.  The mean field theory for image motion estimation , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[29]  Ali Mohammad-Djafari,et al.  On the estimation of hyperparameters in Bayesian approach of solving inverse problems , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[30]  B. Gidas,et al.  A Variational Method for Estimating the Parameters of MRF from Complete or Incomplete Data , 1993 .

[31]  Gerasimos Potamianos,et al.  Partition function estimation of Gibbs random field images using Monte Carlo simulations , 1993, IEEE Trans. Inf. Theory.

[32]  Simon R. Cherry,et al.  Fast gradient-based methods for Bayesian reconstruction of transmission and emission PET images , 1994, IEEE Trans. Medical Imaging.

[33]  Jun Zhang,et al.  Maximum-likelihood parameter estimation for unsupervised stochastic model-based image segmentation , 1994, IEEE Trans. Image Process..

[34]  J. Besag,et al.  Bayesian Computation and Stochastic Systems , 1995 .

[35]  E U Mumcuoğlu,et al.  Bayesian reconstruction of PET images: methodology and performance analysis. , 1996, Physics in medicine and biology.