Characterizing the sparseness of neural codes

It is often suggested that efficient neural codes for natural visual information should be `sparse'. However, the term `sparse' has been used in two different ways - firstly to describe codes in which few neurons are active at any time (`population sparseness'), and secondly to describe codes in which each neuron's lifetime response distribution has high kurtosis (`lifetime sparseness'). Although these ideas are related, they are not identical, and the most common measure of lifetime sparseness - the kurtosis of the lifetime response distributions of the neurons - provides no information about population sparseness. We have measured the population sparseness and lifetime kurtosis of several biologically inspired coding schemes. We used three measures of population sparseness (population kurtosis, Treves-Rolls sparseness and `activity sparseness'), and found them to be in close agreement with one another. However, we also measured the lifetime kurtosis of the cells in each code. We found that lifetime kurt...

[1]  D. H. Hubel,et al.  RECEPTIVE FIELDS, BINOCULAR AND FUNCTIONAL ARCHITECTURE IN THE CAT’S VISUAL CORTEX , 1962 .

[2]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[3]  D. Hubel,et al.  Sequence regularity and geometry of orientation columns in the monkey striate cortex , 1974, The Journal of comparative neurology.

[4]  J. Movshon,et al.  Spatial summation in the receptive fields of simple cells in the cat's striate cortex. , 1978, The Journal of physiology.

[5]  J. Daugman.  Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[6]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[7]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[8]  D. Field,et al.  The structure and symmetry of simple-cell receptive-field profiles in the cat’s visual cortex , 1986, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[9]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[10]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[11]  G. J. Burton,et al.  Color and spatial structure in natural scenes. , 1987, Applied optics.

[12]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[13]  D. Tolhurst.  The amount of information transmitted about contrast by neurones in the cat's visual cortex , 1989, Visual Neuroscience.

[14]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[15]  Bernhard Wegmann,et al.  Statistical dependence between orientation filter outputs used in a human-vision-based image code , 1990, Other Conferences.

[16]  Edmund T. Rolls,et al.  What determines the capacity of autoassociative memories in the brain? Network , 1991 .

[17]  Leslie S. Smith,et al.  The principal components of natural images , 1992 .

[18]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[19]  D. Tolhurst,et al.  Amplitude spectra of natural images , 1992 .

[20]  I. Ohzawa,et al.  Spatiotemporal organization of simple-cell receptive fields in the cat's striate cortex. I. General characteristics and postnatal development. , 1993, Journal of neurophysiology.

[21]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[22]  E T Rolls,et al.  Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. , 1995, Journal of neurophysiology.

[23]  C. Fyfe,et al.  Finding compact and sparse-distributed representations of visual images , 1995 .

[24]  R. Baddeley,et al.  Searching for filters with 'interesting' output distributions: an uninteresting direction to explore? , 1996, Network.

[25]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[26]  William B. Levy,et al.  Energy Efficient Neural Codes , 1996, Neural Computation.

[27]  R W Prager,et al.  Development of low entropy coding in a recurrent network. , 1996, Network.

[28]  L. Abbott,et al.  Responses of neurons in primary and inferior temporal visual cortices to natural scenes , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[29]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[30]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[31]  Gabriel Cristóbal,et al.  Image Representation with Gabor Wavelets and Its Applications , 1997 .

[32]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[33]  D. Ruderman,et al.  Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[34]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[35]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[36]  B Willmore,et al.  A Comparison of Natural-Image-Based Models of Simple-Cell Coding , 2000, Perception.

[37]  Sparseness and kurtosis of computational models of simple-cell coding in primary visual cortex , 2000 .