Nonlinear image processing using artificial neural networks

Publisher Summary This chapter discusses the application of neural networks in image processing. Many researchers have attempted to apply artificial neural networks (ANNs) to image processing problems. It is an overview of what can now perhaps be called the "neural network hype" in image processing. In some of these applications the most interesting aspect of ANNs, the fact that they can be trained, was not (or only partly) used. This held especially for applications to the first few tasks in the image processing chain: preprocessing and feature extraction. Another advantage of ANNs often used to justify their use is the ease of hardware implementation; however, in most publications, this did not seem to be the reason for application. The experiment on supervised classification, in handwritten digit recognition, showed that ANNs are quite capable of solving difficult object recognition problems. A number of ANN architectures were trained to mimic the Kuwahara filter, a nonlinear edge-preserving smoothing filter used in preprocessing. The experiments showed that careful construction of the training set is very important. ANNs seem to be most applicable for problems requiring a nonlinear solution, for which there is a clear, unequivocal performance criterion.

[1]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[2]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[3]  M. Egmont-Petersen,et al.  An explanation facility for a neural network trained to predict atrial fibrillation directly after cardiac surgery , 1998, Computers in Cardiology 1998. Vol. 25 (Cat. No.98CH36292).

[4]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[5]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[6]  I. Guyon,et al.  Handwritten digit recognition: applications of neural network chips and automatic learning , 1989, IEEE Communications Magazine.

[7]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[8]  Sarunas Raudys,et al.  Evolution and generalization of a single neurone: I. Single-layer perceptron as seven statistical classifiers , 1998, Neural Networks.

[9]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[10]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[11]  Eduardo D. Sontag,et al.  Feedback Stabilization Using Two-Hidden-Layer Nets , 1991, 1991 American Control Conference.

[12]  Leonid I. Perlovsky,et al.  Conundrum of Combinatorial Complexity , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[14]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[15]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[16]  Yann LeCun,et al.  Constrained neural networks for pattern recognition , 1991 .

[17]  Yoshua Bengio,et al.  Neural networks for speech and sequence recognition , 1996 .

[18]  John C. Platt,et al.  A Convolutional Neural Network Hand Tracker , 1994, NIPS.

[19]  Arie Hasman,et al.  Assessing the importance of features for multi-layer perceptrons , 1998, Neural Networks.

[20]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[21]  T. Poggio,et al.  Ill-posed problems in early vision: from computational theory to analogue networks , 1985, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[22]  Arnold W. M. Smeulders,et al.  BESSI: an experimentation system for vision module evaluation , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[23]  Vijay K. Madisetti,et al.  The Digital Signal Processing Handbook , 1997 .

[24]  William H. Press,et al.  Numerical recipes in C , 2002 .

[25]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[28]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[29]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[30]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[31]  Dick de Ridder,et al.  Adaptive methods of image processing , 2001 .

[32]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[33]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[34]  R. Duin,et al.  A weight set decorrelating algorithm for neural network interpretation and symmetry breaking , 1999 .

[35]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[36]  Michael Egmont-Petersen,et al.  Image processing with neural networks - a review , 2002, Pattern Recognit..

[37]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[38]  Robert P. W. Duin,et al.  A new measure for the effect of sharpening and smoothing filters on images , 1999 .

[39]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[40]  PAUL D. GADER,et al.  Segmentation free shared weight networks for automatic vehicle detection , 1995, Neural Networks.

[41]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[42]  Kendall Preston,et al.  Digital processing of biomedical images , 1976 .

[43]  Isabelle Guyon,et al.  On-line cursive script recognition using time-delay neural networks and hidden Markov models , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[45]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[46]  Lieuwe Jan Spreeuwers,et al.  Image Filtering with Neural Networks: Applications and performance evaluation , 1992 .

[47]  David B. Fogel.  An information criterion for optimal neural network selection , 1991, IEEE Trans. Neural Networks.

[48]  C. D. Green,et al.  Are Connectionist Models Theories of Cognition , 1998 .

[49]  van den Maarten Berg,et al.  COMPUTERS IN CARDIOLOGY 1995 , 1995 .

[50]  Rudy Setiono,et al.  Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting , 1997, Neural Computation.

[51]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[52]  Robert M. Haralick,et al.  Performance Characterization in Computer Vision , 1993, BMVC.

[53]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[54]  Nikola Kasabov,et al.  Brain-like Computing and Intelligent Information Systems , 1998 .

[55]  Richard Lippmann,et al.  Neural Network Classifiers Estimate Bayesian a posteriori Probabilities , 1991, Neural Computation.

[56]  Pietro Burrascano,et al.  A norm selection criterion for the generalized delta rule , 1991, IEEE Trans. Neural Networks.

[57]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[58]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[59]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[60]  Dick de Ridder,et al.  Shared Weights Neural Networks in Image Analysis , 1996 .

[61]  Leon Sterling,et al.  AI '92 : proceedings of the 5th Australian Joint Conference on Artificial Intelligence : Hobart, Tasmania, 16-18 November 1992 , 1992 .

[62]  Jack Perkins,et al.  Pattern recognition in practice , 1980 .

[63]  Huan Liu,et al.  Neural-network feature selector , 1997, IEEE Trans. Neural Networks.

[64]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[65]  Hiroshi Katsulai,et al.  Evaluation of Image Fidelity by Means of the Fidelogram and Level Mean-Square Error , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Yann LeCun,et al.  Efficient Pattern Recognition Using a New Transformation Distance , 1992, NIPS.

[67]  Jordan B. Pollack,et al.  Exact representations from feed-forward networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[68]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[69]  Min-Seok Choi,et al.  A novel two stage template matching method for rotation and illumination invariance , 2002, Pattern Recognit..

[70]  Yoshua Bengio,et al.  Globally trained handwritten word recognizer using spatial representation, space displacement neural networks and hidden Markov models , 1993 .

[71]  E. R. Davies,et al.  Relative effectiveness of neural networks for image noise suppression , 1994 .

[72]  Françoise Fogelman-Soulié,et al.  Multi-Modular Neural Network Architectures: Applications in Optical Character and Human Face Recognition , 1993, Int. J. Pattern Recognit. Artif. Intell..

[73]  Veljko Milutinovic,et al.  Neural Networks: Concepts, Applications, and Implementations , 1991 .

[74]  Sarunas Raudys,et al.  Evolution and generalization of a single neurone: : II. Complexity of statistical classifiers and sample size considerations , 1998, Neural Networks.

[75]  R. Duin,et al.  A weight set decorrelating training algorithm for neural network interpretation and symmetry breaking , 1999 .