Analysis of the two-dimensional receptive fields learned by the Generalized Hebbian Algorithm in response to random input

The Generalized Hebbian Algorithm has been proposed for training linear feedforward neural networks and has been proven to cause the weights to converge to the eigenvectors of the input distribution (Sanger 1989a, b). For an input distribution given by 2D Gaussian smoothed white noise inside a Gaussian window, some of the masks learned by the Generalized Hebbian Algorithm resemble edge and bar detectors. Since these do not match the form of the actual eigenvectors of this distribution (Linsker 1987, 1990), we seek an explanation of the development of the masks prior to complete convergence to the correct solution. Analysis in the spatial and spatial frequency domains sheds light on this development, and shows that the masks which occur tend to be localized in the spatial frequency domain, reminiscent of one of the properties of 2D Gabor filters proposed by Daugman ( 1980, 1985) as a model for the receptive fields of cells in primate visual cortex.

[1]  Terence D. Sanger.  Optimal Unsupervised Learning in Feedforward Neural Networks , 1989 .

[2]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[3]  A. Yuille,et al.  Spontaneous symmetry-breaking energy functions and the emergence of orientation selective cortical cells , 2004, Biological Cybernetics.

[4]  Ralph Linsker,et al.  Computer simulation in brain science: Development of feature-analyzing cells and their columnar organization in a layered self-adaptive network , 1988 .

[5]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[6]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[7]  Daniel A. Pollen,et al.  Visual cortical neurons as localized spatial frequency filters , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  D. Hubel.  Exploration of the primary visual cortex, 1955–78 , 1982, Nature.

[9]  Terence D. Sanger,et al.  Optimal unsupervised learning , 1988, Neural Networks.

[10]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[11]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[12]  D. M. Kammen,et al.  Quadrature and the development of orientation selective cortical cells by Hebb rules , 1989, Biological Cybernetics.

[13]  S.-I. Amari,et al.  Neural theory of association and concept-formation , 1977, Biological Cybernetics.

[14]  Ralph Linsker,et al.  Towards an Organizing Principle for a Layered Perceptual Network , 1987, NIPS.

[15]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[16]  D. M. Kammen,et al.  Spontaneous symmetry-breaking energy functionsand the emergence of orientation selective cortical cells , 1988, Biological Cybernetics.

[17]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[18]  D. Pollen,et al.  Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey. , 1985, The Journal of physiology.

[19]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[20]  John Daugman,et al.  Six formal properties of two-dimensional anisotropie visual filters: Structural principles and frequency/orientation selectivity , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[21]  J. Daugman.  Spatial visual channels in the fourier plane , 1984, Vision Research.

[22]  D. Pollen,et al.  Relationship between spatial frequency selectivity and receptive field profile of simple cells. , 1979, The Journal of physiology.

[23]  D. Kazakos.  Optimal constrained representation and filtering of signals , 1983 .

[24]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[25]  D. Pollen,et al.  Phase relationships between adjacent simple cells in the visual cortex. , 1981, Science.

[26]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[27]  J. Daugman.  Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[28]  Dana H. Ballard,et al.  Learning receptive fields using predictive feedback , 2006, Journal of Physiology-Paris.

[29]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[30]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[31]  L. Maffei,et al.  The visual cortex as a spatial frequency analyser. , 1973, Vision research.

[32]  Dennis Gabor,et al.  Theory of communication , 1946 .

[33]  Gene H. Golub,et al.  Matrix computations , 1983 .

[34]  P. Lennie,et al.  Spatial frequency analysis in the visual system. , 1985, Annual review of neuroscience.

[35]  R. Linsker,et al.  From basic network principles to neural architecture , 1986 .