Information recovery from rank-order encoded images

The work described in this paper is inspired by SpikeNET, a system developed to test the feasibility of using rank-order codes in modelling largescale networks of asynchronously spiking neurons. The rank-order code theory proposed by Thorpe concerns the encoding of information by a population of spiking neurons in the primate visual system. The theory proposes using the order of firing across a network of asynchronously firing spiking neurons as a neural code for information transmission. In this paper we aim to measure the perceptual similarity between the image input to a model retina, based on that originally designed and developed by VanRullen and Thorpe, and an image reconstructed from the rank-order encoding of the input image. We use an objective metric originally proposed by Petrovic to estimate perceptual edge preservation in image fusion which, after minor modifications, is very much suited to our purpose. The results show that typically 75% of the edge information of the input stimulus is retained in the reconstructed image, and we show how the available information increases with successive spikes in the rank-order code.

[1]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[2]  D. Dacey.  The mosaic of midget ganglion cells in the human retina , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  J. Atick,et al.  STATISTICS OF NATURAL TIME-VARYING IMAGES , 1995 .

[4]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[5]  Scott Daly,et al.  Digital Images and Human Vision , 1993 .

[6]  Pramod K. Varshney,et al.  A human perception inspired quality metric for image fusion based on regional information , 2007, Inf. Fusion.

[7]  S. Thorpe,et al.  1 BIOLOGICAL CONSTRAINTS ON CONNECTIONIST MODELLING , 1989 .

[8]  Alexander Toet,et al.  Perceptual evaluation of different image fusion schemes , 2003 .

[9]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[10]  John E. Dowling.  Neurons and Networks: An Introduction to Neuroscience , 1992 .

[11]  Refractor.  Vision , 2000, The Lancet.

[12]  R. Johansson,et al.  First spikes in ensembles of human tactile afferents code complex spatial fingertip events , 2004, Nature Neuroscience.

[13]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[14]  Barney M. Berlin,et al.  Size , 1989, Encyclopedia of Evolutionary Psychological Science.

[15]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[16]  R. Masland.  The fundamental plan of the retina , 2001, Nature Neuroscience.

[17]  Zhaoping Li.  Different Retinal Ganglion Cells have Different Functional Goals , 1992, Int. J. Neural Syst..

[18]  Jacques Gautrais,et al.  SpikeNET: A simulator for modeling large networks of integrate and fire neurons , 1999, Neurocomputing.

[19]  E. D. Adrian,et al.  The Basis of Sensation , 1928, The Indian Medical Gazette.

[20]  Zhou Wang,et al.  Why is image quality assessment so difficult? , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[21]  D. Field,et al.  Natural image statistics and efficient coding. , 1996, Network.

[22]  H. K. Hartline,et al.  INHIBITION IN THE EYE OF LIMULUS , 1956, The Journal of general physiology.

[23]  Stéphane Mallat,et al.  Matching pursuit of images , 1995, Proceedings., International Conference on Image Processing.

[24]  C. Blackstone.  The Neuron: Cell and Molecular Biology , 2003 .

[25]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[26]  Rufin van Rullen,et al.  SpikeNet: real-time visual processing with one spike per neuron , 2004, Neurocomputing.

[27]  S. Thorpe,et al.  Speed of processing in the human visual system , 1996, Nature.

[28]  M. Diamond,et al.  The Role of Spike Timing in the Coding of Stimulus Location in Rat Somatosensory Cortex , 2001, Neuron.

[29]  Vladimir Petrovic,et al.  Objective evaluation of signal-level image fusion performance , 2005 .

[30]  Vladimir S. Petrovic,et al.  Subjective tests for image fusion evaluation and objective metric validation , 2007, Inf. Fusion.

[31]  D. Dacey,et al.  Dendritic field size and morphology of midget and parasol ganglion cells of the human retina. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Simon J Thorpe,et al.  SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons , 2003, Network.

[33]  Gerhard W. Cibis,et al.  Clinical Anatomy of the Eye , 1990 .

[34]  M. Land.  Eye Movements in Daily Life , 2003 .

[35]  Peter G. J. Barten,et al.  Contrast sensitivity of the human eye and its e ects on image quality , 1999 .

[36]  Michael J. Berry,et al.  The Neural Code of the Retina , 1999, Neuron.

[37]  William H. Press,et al.  Numerical recipes in C , 2002 .

[38]  B. Boycott,et al.  Retinal ganglion cell density and cortical magnification factor in the primate , 1990, Vision Research.

[39]  Stephen B. Furber,et al.  Maximising information recovery from rank-order codes , 2007, SPIE Defense + Commercial Sensing.

[40]  Jacques Gautrais,et al.  Rapid Visual Processing using Spike Asynchrony , 1996, NIPS.

[41]  B J Melloni,et al.  How the retina works. , 1971, American Family Physician.

[42]  Y. Petrov,et al.  Local correlations, information redundancy, and sufficient pixel depth in natural images. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  Li Zhaoping,et al.  The distribution of visual objects on the retina: connecting eye movements and cone distributions. , 2003, Journal of vision.

[44]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.

[45]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[46]  Michael T. Heath,et al.  Scientific Computing: An Introductory Survey , 1996 .

[47]  S. A. Talbot.  Physiology of the retina and the visual pathway , 1961 .

[48]  Jacques Gautrais,et al.  Rank order coding , 1998 .

[49]  Steve Furber,et al.  Information recovery from rank-order encoded images , 2006 .

[50]  Li Zhaoping,et al.  Theoretical understanding of the early visual processes by data compression and data selection , 2006, Network.

[51]  Rogério Melo Kinape,et al.  A study of the most important image quality measures , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[52]  S. Thorpe,et al.  Spike times make sense , 2005, Trends in Neurosciences.

[53]  R. Hetherington.  The Perception of the Visual World , 1952 .

[54]  Alexander Toet,et al.  Performance comparison of different gray-level image fusion schemes through a universal image quality index , 2003, SPIE Defense + Commercial Sensing.

[55]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[56]  S. Thorpe,et al.  A Limit to the Speed of Processing in Ultra-Rapid Visual Categorization of Novel Natural Scenes , 2001, Journal of Cognitive Neuroscience.

[57]  D Kersten,et al.  Predictability and redundancy of natural images. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[58]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[59]  P. Sterling,et al.  How Much the Eye Tells the Brain , 2006, Current Biology.

[60]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[61]  Vladimir S. Petrovic,et al.  Evaluation of Image Fusion Performance with Visible Differences , 2004, ECCV.

[62]  B. Boycott,et al.  Functional architecture of the mammalian retina. , 1991, Physiological reviews.

[63]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[64]  S. W. Kuffler.  Discharge patterns and functional organization of mammalian retina. , 1953, Journal of neurophysiology.

[65]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[66]  Bruno A. Olshausen,et al.  Vision and the Coding of Natural Images , 2000, American Scientist.

[67]  Simon J. Thorpe,et al.  Ultra-Rapid Scene Categorization with a Wave of Spikes , 2002, Biologically Motivated Computer Vision.

[68]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[69]  M. Newman.  Power laws, Pareto distributions and Zipf's law , 2005 .

[70]  H. Kolb,et al.  The midget pathways of the primate retina , 2004, Documenta Ophthalmologica.

[71]  Michael J. Berry,et al.  Redundancy in the Population Code of the Retina , 2005, Neuron.

[72]  J. V. van Hateren,et al.  Real and optimal neural images in early vision , 1992, Nature.

[73]  R. W. Rodieck.  The First Steps in Seeing , 1998 .

[74]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[75]  Pavel Slavík,et al.  Evaluation of two principal approaches to objective image quality assessment , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[76]  Henk J. A. M. Heijmans,et al.  A new quality metric for image fusion , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[77]  J Gautrais,et al.  Rate coding versus temporal order coding: a theoretical approach. , 1998, Bio Systems.

[78]  R. W. Rodieck.  Quantitative analysis of cat retinal ganglion cell response to visual stimuli. , 1965, Vision research.

[79]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[80]  Vladimir S. Petrovic,et al.  Objective pixel-level image fusion performance measure , 2000, SPIE Defense + Commercial Sensing.