Genetic design of biologically inspired receptive fields for neural pattern recognition

This paper proposes a new method for the design, through simulated evolution, of biologically inspired receptive fields in feedforward neural networks (NNs). The method is intended to enhance pattern recognition performance by creating new neural architectures specifically tuned for a particular pattern recognition problem. It proposes a combined neural architecture composed of two networks in cascade: a feature extraction network (FEN) followed by a neural classifier. The FEN is composed of several layers with receptive fields constructed by additive superposition of excitatory and inhibitory fields. A genetic algorithm (GA) is used to select receptive field parameters to improve classification performance. The parameters are receptive field size, orientation, and bias as well as the number of different receptive fields in each layer. Based on a random initial population where each individual represents a different neural architecture, the GA creates new enhanced individuals. The method is applied to handwritten digit classification and face recognition. In both problems, results show strong dependency between NN classification performance and receptive field architecture. GA selected parameters of the receptive fields produced improvements in the classification performance on the test set up to 90.8% for the problem of handwritten digit classification and up to 84.2% for the face recognition problem. On the same test sets, results were compared advantageously to standard feedforward multilayer perceptron (MLP) NNs where receptive fields are not explicitly defined. The MLP reached a maximum classification performance of 84.9% and 77.5% in both problems, respectively.

[1]  Donald E. Brown,et al.  Fast generic selection of features for neural network classifiers , 1992, IEEE Trans. Neural Networks.

[2]  F. Busetti.  Genetic algorithms overview , 2000 .

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  John Mark Bishop,et al.  A comparison of fast training algorithms over two real problems , 1997 .

[5]  Korris Fu-Lai Chung,et al.  Neocognitron based handwriting recognition system performance tuning using genetic algorithm , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[6]  Chris Matthews,et al.  An Evolutionary Approach to the Specification of High Performing Backpropagation Neural Networks , 1997, ICONIP.

[7]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[8]  Frédéric Alexandre,et al.  The cortical column: A new processing unit for multilayered networks , 1991, Neural Networks.

[9]  Stéphane Mallat,et al.  Multifrequency Channel Decompositions of Images , 1989 .

[10]  D. B. Fogel,et al.  Evolving neural networks , 1990, Biological Cybernetics.

[11]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[12]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[13]  Claus Nebauer,et al.  Evaluation of convolutional neural networks for visual recognition , 1998, IEEE Trans. Neural Networks.

[14]  Smith Ge,et al.  The Cortical Representation of the Macula. , 1930 .

[15]  RussLL L. Ds Vnlos,et al.  SPATIAL FREQUENCY SELECTIVITY OF CELLS IN MACAQUE VISUAL CORTEX , 2022 .

[16]  J. P. Thomas,et al.  Neural recoding in human pattern vision: model and mechanisms , 1999, Vision Research.

[17]  Sid Deutsch,et al.  Visual Pattern Recognition, Neural Networks, and Household Chores , 1993 .

[18]  Stéphane Mallat,et al.  Multifrequency channel decompositions of images and wavelet models , 1989, IEEE Trans. Acoust. Speech Signal Process..

[19]  John J. Grefenstette,et al.  Proportional selection and sampling algorithms , 1997 .

[20]  Takayuki Ito,et al.  Neocognitron: A neural network model for a mechanism of visual pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[21]  David Lovell,et al.  The neocognitron as a system for handwritten character recognition : limitations and improvements , 1994 .

[22]  F R Adams,et al.  A parallel network for visual cognition , 1992, IEEE Trans. Neural Networks.

[23]  Alexander Dimitrov,et al.  Spatial Decorrelation in Orientation-Selective Cortical Cells , 1998, Neural Computation.

[24]  D. Hubel.  Exploration of the primary visual cortex, 1955–78 , 1982, Nature.

[25]  M. J. Wright.  Training and Testing of Neural Net Window Operators on Spatiotemporal Image Sequences , 1992 .

[26]  Yehoshua Y. Zeevi,et al.  Reorganization and diversification of signals in vision , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[27]  Claudio A. Perez,et al.  Designing biologically inspired receptive fields for neural pattern recognition technology , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[28]  Melanie Mitchell,et al.  Genetic algorithms: An overview , 1995, Complex..

[29]  R. Näsänen.  Spatial frequency bandwidth used in the recognition of facial images , 1999, Vision Research.

[30]  C.A. Perez,et al.  Handwritten digit recognition by a neural network with slope detection and multiple planes per layer architecture , 1993, Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Societ.

[31]  Terrence J. Sejnowski,et al.  Relation Between Retinotopical and Orientation Maps in Visual Cortex , 1999, Neural Computation.

[32]  C.A. Perez,et al.  Improvements on handwritten digit recognition by genetic selection of neural network topology and by augmented training , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[33]  Risto Miikkulainen,et al.  Evolving Neural Networks to Play Go , 2004, Applied Intelligence.

[34]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[35]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[36]  A. Watson,et al.  A hexagonal orthogonal-oriented pyramid as a model of image representation in visual cortex , 1989, IEEE Transactions on Biomedical Engineering.

[37]  D.J. Granrath,et al.  The role of human visual models in image processing , 1981, Proceedings of the IEEE.

[38]  C.A. Perez,et al.  Genetic selection of biologically inspired receptive fields for computational vision , 1999, Proceedings of the First Joint BMES/EMBS Conference. 1999 IEEE Engineering in Medicine and Biology 21st Annual Conference and the 1999 Annual Fall Meeting of the Biomedical Engineering Society (Cat. N.

[39]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[40]  D H HUBEL,et al.  THE VISUAL CORTEX OF THE BRAIN. , 1963, Scientific American.

[41]  S. W. Kuffler.  Discharge patterns and functional organization of mammalian retina. , 1953, Journal of neurophysiology.

[42]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[43]  C. Fyfe,et al.  Finding compact and sparse-distributed representations of visual images , 1995 .

[44]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[45]  Takeo Watanabe,et al.  The primary visual cortex fills in color , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[46]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[47]  Guillermo Sapiro,et al.  Creating connected representations of cortical gray matter for functional MRI visualization , 1997, IEEE Transactions on Medical Imaging.

[48]  Graham M. Megson,et al.  Implementing a generic systolic array for genetic algorithms , 1997 .

[49]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[50]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[51]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..