Image Restoration on the Hopfield Neural Network

Publisher Summary There are practical advantages for considering the image restoration or superresolution problem in terms of a neural network formalism. An advantage that has been found is the improved performance with respect to ill-conditioning difficulties. There is a large body of empirical evidence that the neural network approach enlarges the basins of attraction of the energy function minima, thus, enhancing the chances of finding better solutions and making the final solution less dependent on the starting parameters. This chapter explains the way in which both binary (two-state) and nonbinary image reconstruction algorithms can be implemented on very similar (Hopfield) neural architectures. The image restoration algorithms discussed in the chapter were originally aimed at achieving performance beyond the diffraction limit, but are in fact capable of compensating simultaneously or separately for aberrations induced by the optical components and for the limitations of the detector. The chapter also describes some image restoration or superresolution algorithms that can be implemented on an artificial neural network. Image restoration methods are well known to be illconditioned, hence, there is the need to employ regularization techniques.

[1]  P. Hansen.  The truncatedSVD as a method for regularization , 1987 .

[2]  Michael A. Fiddy,et al.  Regularized image reconstruction using SVD and a neural network method for matrix inversion , 1993, IEEE Trans. Signal Process..

[3]  Paolo Carnevali,et al.  Image Processing by Simulated Annealing , 1985, IBM J. Res. Dev..

[4]  B. K. Jenkins,et al.  Photonic Implementations of Neural Networks , 1992, Optical Computing.

[5]  L. Jones.  Approximation-theoretic derivation of logarithmic entropy principles for inverse problems and unique extension of the maximum entropy method to incorporate prior knowledge , 1989 .

[6]  G Eichmann,et al.  Superresolving signal and image restoration using a linear associative memory. , 1987, Applied optics.

[7]  G. J. Dunning,et al.  Holographic associative memory with nonlinearities in the correlation domain. , 1987, Applied optics.

[8]  R. Gerchberg.  Super-resolution through Error Energy Reduction , 1974 .

[9]  L. Jones.  On a conjecture of Huber concerning the convergence of projection pursuit regression , 1987 .

[10]  M. Fiddy,et al.  High-resolution image reconstruction based on matrix inversion on a fully connected architecture , 1990 .

[11]  Takeshi Sakata,et al.  A single 1.5-V digital chip for a 106 synapse neural network , 1993, IEEE Trans. Neural Networks.

[12]  Harbans S. Dhadwal,et al.  Regularized Iterative and Non-iterative Procedures for Object Restoration from Experimental Data , 1983 .

[13]  Ramdas Kumaresan,et al.  Array signal processing with interconnected Neuron-like elements , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  J B. Abbiss,et al.  Super-Resolution And Neural Computing , 1988, Photonics West - Lasers and Applications in Science and Engineering.

[15]  Michael A. Fiddy,et al.  Stable, noniterative object reconstruction from incomplete data using a priori knowledge , 1983 .

[16]  G. Wahba,et al.  Generalized Inverses in Reproducing Kernel Spaces: An Approach to Regularization of Linear Operator Equations , 1974 .

[17]  D. Z. Anderson,et al.  Bistable ring resonator utilizing saturable photorefractive gain and loss. , 1989, Optics letters.

[18]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[19]  Arthur E. T. Chiou,et al.  Photorefractive Nonlinear Optics And Optical Computing , 1989 .

[20]  M Cronin-Golomb,et al.  Ring self-pumped phase conjugator using total internal reflection in photorefractive strontium barium niobate. , 1989, Optics letters.

[21]  On extending the orthogonality property of minimum norm solutions in Hilbert space to general methods for linear inverse problems , 1990 .

[22]  D. Brady,et al.  Adaptive optical networks using photorefractive crystals. , 1988, Applied optics.

[23]  Robert J. Marks,et al.  Iterative Coherent Processor For Bandlimited Signal Extrapolation , 1980, Other Conferences.

[24]  P. Carnevali,et al.  Image processing by stimulated annealing , 1985 .

[25]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inverse problems , 1990 .

[26]  G Indebetouw,et al.  Iterative image processing using a cavity with a phase-conjugate mirror. , 1992, Applied optics.

[27]  J. Anthony Gualtieri,et al.  Solving inversion problems with neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[28]  R. Chellappa,et al.  Image restoration with neural networks , 1992 .

[29]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Yuri Owechko,et al.  Nonlinear holographic associative memories , 1989 .

[31]  L. Jones,et al.  Computationally feasible high-resolution minimum-distance procedures which extend the maximum-entropy method , 1989 .

[32]  P. Yeh,et al.  Optical interconnection using photorefractive dynamic holograms. , 1988, Applied optics.

[33]  Robert J. Marks,et al.  Gerchberg’s extrapolation algorithm in two dimensions , 1981 .

[34]  I. Johnstone,et al.  Projection-Based Approximation and a Duality with Kernel Methods , 1989 .

[35]  J. P. Burg,et al.  Maximum entropy spectral analysis. , 1967 .

[36]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[37]  Charles L. Byrne,et al.  Image restoration and resolution enhancement , 1983 .

[38]  Michael A. Fiddy,et al.  Superresolution algorithms for a modified Hopfield neural network , 1991, IEEE Trans. Signal Process..

[39]  H. J. White,et al.  Digital and analogue holographic associative memories , 1988 .

[40]  N. Farhat,et al.  Radar image reconstruction based on neural net models , 1988, 1988 IEEE AP-S. International Symposium, Antennas and Propagation.

[41]  Michael A. Fiddy,et al.  On The Application Of Neural Networks To The Solution Of Image Restoration Problems , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[42]  D Z Anderson,et al.  Dynamic optical interconnects: volume holograms as optical two-port operators. , 1987, Applied optics.

[43]  Charles L. Byrne,et al.  General entropy criteria for inverse problems, with applications to data compression, pattern classification, and cluster analysis , 1990, IEEE Trans. Inf. Theory.

[44]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  B. K. Jenkins,et al.  Image restoration using a neural network , 1988, IEEE Trans. Acoust. Speech Signal Process..

[46]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[47]  Durga P. Panda,et al.  Image restoration and enhancement , 1976 .

[48]  Soo-Young Lee,et al.  An Optimization Network for Matrix Inversion , 1987, NIPS.

[49]  John Wawrzynek,et al.  SPERT: a VLIW/SIMD neuro-microprocessor , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[50]  D Psaltis,et al.  Storage capacity of holographic associative memories. , 1986, Optics letters.

[51]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[52]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[53]  M A Fiddy,et al.  Image-restoration algorithms for a fully connected architecture. , 1990, Optics letters.

[54]  M. Bertero,et al.  Linear inverse problems with discrete data: II. Stability and regularisation , 1988 .

[55]  R. Marks.  Coherent optical extrapolation of 2-D band-limited signals: processor theory , 1980 .

[56]  C. R. Rao,et al.  Cross entropy, dissimilarity measures, and characterizations of quadratic entropy , 1985, IEEE Trans. Inf. Theory.

[57]  Shmuel Sternklar,et al.  Photorefractive oscillators , 1989 .

[58]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[59]  B H Soffer,et al.  Associative holographic memory with feedback using phase-conjugate mirrors. , 1986, Optics letters.