A Simple Hebbian/Anti-Hebbian Network Learns the Sparse, Independent Components of Natural Images

Slightly modified versions of an early Hebbian/anti-Hebbian neural network are shown to be capable of extracting the sparse, independent linear components of a prefiltered natural image set. An explanation for this capability in terms of a coupling between two hypothetical networks is presented. The simple networks presented here provide alternative, biologically plausible mechanisms for sparse, factorial coding in early primate vision.

[1]  C. Blakemore,et al.  Lateral Inhibition between Orientation Detectors in the Human Visual System , 1970, Nature.

[2]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[3]  D. G. Albrecht,et al.  Striate cortex of monkey and cat: contrast response function. , 1982, Journal of neurophysiology.

[4]  E. W. Kairiss,et al.  Hebbian synapses: biophysical mechanisms and algorithms. , 1990, Annual review of neuroscience.

[5]  J. Rauschecker,et al.  Mechanisms of visual plasticity: Hebb synapses, NMDA receptors, and beyond. , 1991, Physiological reviews.

[6]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[7]  Peter H. Schiller,et al.  The ON and OFF channels of the visual system , 1992, Trends in Neurosciences.

[8]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[9]  Mohamad H. Hassoun,et al.  Statistical basis of nonlinear hebbian learning and application to clustering , 1995, Neural Networks.

[10]  C. Shatz,et al.  Synaptic Activity and the Construction of Cortical Circuits , 1996, Science.

[11]  Erkki Oja,et al.  Image Feature Extraction Using Independent Component Analysis , 1996 .

[12]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[13]  L. Abbott,et al.  Responses of neurons in primary and inferior temporal visual cortices to natural scenes , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[14]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[15]  J. Deuchars,et al.  Synaptic interactions in neocortical local circuits: dual intracellular recordings in vitro. , 1997, Cerebral cortex.

[16]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[17]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[18]  D. Ruderman,et al.  Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[19]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[20]  M. Carandini,et al.  Visual cortex: Fatigue and adaptation , 2000, Current Biology.

[21]  J. Budd,et al.  Local lateral connectivity of inhibitory clutch cells in layer 4 of cat visual cortex (area 17) , 2001, Experimental Brain Research.

[22]  Allan Gottschalk,et al.  Derivation of the Visual Contrast Response Function by Maximizing Information Rate , 2002, Neural Computation.

[23]  G. Boynton,et al.  Orientation-Specific Adaptation in Human Visual Cortex , 2003, The Journal of Neuroscience.

[24]  Patrik O. Hoyer,et al.  Modeling Receptive Fields with Non-Negative Sparse Coding , 2002, Neurocomputing.

[25]  P. Lennie.  The Cost of Cortical Computation , 2003, Current Biology.

[26]  P. Földiák,et al.  Forming sparse representations by local anti-Hebbian learning , 1990, Biological Cybernetics.

[27]  D. Badcock,et al.  Implicit exploitation of regularities: Novel correlations in images quickly alter visual perception , 2006, Vision Research.

[28]  Jochen Triesch,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[29]  RussLL L. Ds Vnlos,et al.  SPATIAL FREQUENCY SELECTIVITY OF CELLS IN MACAQUE VISUAL CORTEX , 2022 .