Error-correcting codes and image restoration with multiple stages of dynamics

We consider the problems of error-correcting codes and image restoration with multiple stages of dynamics. Information extracted from the former stage can be used selectively to improve the performance of the latter one. Analytic results were derived for the mean-field systems using the cavity method. We find that it has the advantage of being tolerant to uncertainties in hyperparameter estimation, as confirmed by simulations.

[1]  H. Nishimori,et al.  Statistical mechanics of image restoration and error-correcting codes. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[2]  Yoshiyuki Kabashima,et al.  Belief propagation vs. TAP for decoding corrupted messages , 1998 .

[3]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[4]  K Y M Wong,et al.  Improving pattern reconstruction in neural networks by activity dynamics , 1996 .

[5]  Zhenyu Zhou,et al.  Approximate maximum likelihood hyperparameter estimation for Gibbs priors , 1995, Proceedings., International Conference on Image Processing.

[6]  Y. M. WongDepartment,et al.  Microscopic Equations and Stability Conditions in Optimal Neural Networks , 1995 .

[7]  Jonathan Michael Pryce.  Statistical mechanics of image restoration , 1995 .

[8]  H. Nishimori.  Optimum Decoding Temperature for Error-Correcting Codes , 1993 .

[9]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[10]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[11]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Robert J. McEliece,et al.  The Theory of Information and Coding , 1979 .

[13]  D. Saad,et al.  Statistical mechanics of error-correcting codes , 1999 .

[14]  M. Hasselmo,et al.  Gaussian Processes for Regression , 1995, NIPS.

[15]  Mark J van Rossum.  Neural Computation , 1989, Artificial Intelligence.

[16]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[17]  Robert J. McEliece,et al.  The theory of information and coding : a mathematical framework for communication , 1977 .

[18]  G. Barnard.  The Theory of Information , 1951 .

[19]  To be published in: Advances in Neural Information Processing Systems 9, , 2022 .