Characterizing the sparseness of neural codes

It is often suggested that efficient neural codes for natural visual information should be `sparse'. However, the term `sparse' has been used in two different ways - firstly to describe codes in which few neurons are active at any time (`population sparseness'), and secondly to describe codes in which each neuron's lifetime response distribution has high kurtosis (`lifetime sparseness'). Although these ideas are related, they are not identical, and the most common measure of lifetime sparseness - the kurtosis of the lifetime response distributions of the neurons - provides no information about population sparseness. We have measured the population sparseness and lifetime kurtosis of several biologically inspired coding schemes. We used three measures of population sparseness (population kurtosis, Treves-Rolls sparseness and `activity sparseness'), and found them to be in close agreement with one another. However, we also measured the lifetime kurtosis of the cells in each code. We found that lifetime kurtosis is uncorrelated with population sparseness for the codes we used. Lifetime kurtosis is not, therefore, a useful measure of the population sparseness of a code. Moreover, the Gabor-like codes, which are often assumed to have high population sparseness (since they have high lifetime kurtosis), actually turned out to have rather low population sparseness. Surprisingly, principal components filters produced the codes with the highest population sparseness.

[1]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[2]  J. Movshon,et al.  Spatial summation in the receptive fields of simple cells in the cat's striate cortex. , 1978, The Journal of physiology.

[3]  J. Daugman.  Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[4]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[5]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[6]  D. Field,et al.  The structure and symmetry of simple-cell receptive-field profiles in the cat’s visual cortex , 1986, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[7]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[8]  G. J. Burton,et al.  Color and spatial structure in natural scenes. , 1987, Applied optics.

[9]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[10]  D. Tolhurst.  The amount of information transmitted about contrast by neurones in the cat's visual cortex , 1989, Visual Neuroscience.

[11]  Bernhard Wegmann,et al.  Statistical dependence between orientation filter outputs used in a human-vision-based image code , 1990, Other Conferences.

[12]  Edmund T. Rolls,et al.  What determines the capacity of autoassociative memories in the brain? Network , 1991 .

[13]  D. Tolhurst,et al.  Amplitude spectra of natural images. , 1992, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[14]  Leslie S. Smith,et al.  The principal components of natural images , 1992 .

[15]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[16]  I. Ohzawa,et al.  Spatiotemporal organization of simple-cell receptive fields in the cat's striate cortex. I. General characteristics and postnatal development. , 1993, Journal of neurophysiology.

[17]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[18]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[19]  E T Rolls,et al.  Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. , 1995, Journal of neurophysiology.

[20]  C. Fyfe,et al.  Finding compact and sparse-distributed representations of visual images , 1995 .

[21]  R. Baddeley,et al.  Searching for filters with 'interesting' output distributions: an uninteresting direction to explore? , 1996, Network.

[22]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[23]  William B. Levy,et al.  Energy Efficient Neural Codes , 1996, Neural Computation.

[24]  R W Prager,et al.  Development of low entropy coding in a recurrent network. , 1996, Network.

[25]  L. Abbott,et al.  Responses of neurons in primary and inferior temporal visual cortices to natural scenes , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[26]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[27]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[28]  Gabriel Cristóbal,et al.  Image Representation with Gabor Wavelets and Its Applications , 1997 .

[29]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[30]  D. Ruderman,et al.  INDEPENDENT COMPONENT ANALYSIS OF NATURAL IMAGE SEQUENCES YIELDS SPATIOTEMPORAL FILTERS SIMILAR TO SIMPLE CELLS IN PRIMARY VISUAL CORTEX , 1998 .

[31]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[32]  B Willmore,et al.  A Comparison of Natural-Image-Based Models of Simple-Cell Coding , 2000, Perception.

[33]  Sparseness and kurtosis of computational models of simple-cell coding in primary visual cortex , 2000 .