Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

We make an analogy between images and statistical mechanics systems. Pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice-like physical system. The assignment of an energy function in the physical system determines its Gibbs distribution. Because of the Gibbs distribution, Markov random field (MRF) equivalence, this assignment also determines an MRF image model. The energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the MRF. For a range of degradation mechanisms, including blurring, nonlinear deformations, and multiplicative or additive noise, the posterior distribution is an MRF with a structure akin to the image model. By the analogy, the posterior distribution defines another (imaginary) physical system. Gradual temperature reduction in the physical system isolates low energy states (``annealing''), or what is the same thing, the most probable states under the Gibbs distribution. The analogous operation under the posterior distribution yields the maximum a posteriori (MAP) estimate of the image given the degraded observations. The result is a highly parallel ``relaxation'' algorithm for MAP estimation. We establish convergence properties of the algorithm and we experiment with some simple pictures, for which good restorations are obtained at low signal-to-noise ratios.

[1]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .

[2]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[3]  P. Levy.  A Special Problem of Brownian Motion, and a General Theory of Gaussian Random Functions , 1956 .

[4]  Laveen N. Kanal,et al.  Classification of binary random patterns , 1965, IEEE Trans. Inf. Theory.

[5]  P. L. Dobruschin.  The Description of a Random Field by Means of Conditional Probabilities and Conditions of Its Regularity , 1968 .

[6]  F. Spitzer.  Markov Random Fields and Gibbs Ensembles , 1971 .

[7]  B. Frieden.  Restoring with maximum likelihood and maximum entropy. , 1972, Journal of the Optical Society of America.

[8]  John W. Woods,et al.  Two-dimensional discrete Markovian fields , 1972, IEEE Trans. Inf. Theory.

[9]  A. Habibi.  Two-dimensional Bayesian estimate of images , 1972 .

[10]  N. Nahi,et al.  Bayesian recursive image estimation. , 1972 .

[11]  J. Besag.  Nearest‐Neighbour Systems and the Auto‐Logistic Model for Binary Data , 1972 .

[12]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[13]  P. Flinn.  Monte Carlo calculation of phase separation in a two-dimensional Ising system , 1974 .

[14]  M. Bartlett.  The statistical analysis of spatial pattern , 1974, Advances in Applied Probability.

[15]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[16]  Anil K. Jain,et al.  Image Restoration, Modelling, and Reduction of Dimensionality , 1974, IEEE Transactions on Computers.

[17]  A. B. Bortz,et al.  A new algorithm for Monte Carlo simulation of Ising spin systems , 1975 .

[18]  Azriel Rosenfeld,et al.  Scene Labeling by Relaxation Operations , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[19]  D. Griffeath,et al.  Introduction to Random Fields , 2020, 2007.09660.

[20]  J. Stuller,et al.  Two-Dimensional Markov Representations of Sampled Images , 1976, IEEE Trans. Commun..

[21]  Bobby R. Hunt,et al.  Bayesian Methods in Nonlinear Digital Image Restoration , 1977, IEEE Transactions on Computers.

[22]  D. K. Pickard.  A curious binary lattice process , 1977, Journal of Applied Probability.

[23]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[24]  Laveen N. Kanal,et al.  Markov mesh models , 1980 .

[25]  H. Trussell.  The relationship between image restoration by the maximum a posteriori method and a maximum entropy method , 1980 .

[26]  M. Hassner,et al.  The use of Markov Random Fields as models of texture , 1980 .

[27]  A.K. Jain,et al.  Advances in mathematical models for image processing , 1981, Proceedings of the IEEE.

[28]  Azriel Rosenfeld,et al.  Cooperating Processes for Low-Level Vision: A Survey , 1981, Artif. Intell..

[29]  V. Isham.  An Introduction to Spatial Point Processes and Markov Random Fields , 1981 .

[30]  F. R. Hansen,et al.  Image segmentation using simple markov field models , 1982, Comput. Graph. Image Process..

[31]  R. Chellappa,et al.  Digital image restoration using spatial interaction models , 1982 .

[32]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[33]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[36]  David B. Cooper,et al.  Multiple-Window Parallel Adaptive Boundary Finding in Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Rama Chellappa,et al.  Estimation and choice of neighbors in spatial-interaction models of images , 1983, IEEE Trans. Inf. Theory.

[38]  Donald Geman,et al.  Bayes smoothing algorithms for segmentation of images modeled by Markov random fields , 1984, ICASSP.

[39]  Donald Geman,et al.  Application of the Gibbs distribution to image segmentation , 1984, ICASSP.

[40]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[41]  V. Cerný.  Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm , 1985 .