The ripple pond: enabling spiking networks to see

We present the biologically inspired Ripple Pond Network (RPN), a simply connected spiking neural network which performs a transformation converting two dimensional images to one dimensional temporal patterns (TP) suitable for recognition by temporal coding learning and memory networks. The RPN has been developed as a hardware solution linking previously implemented neuromorphic vision and memory structures such as frameless vision sensors and neuromorphic temporal coding spiking neural networks. Working together such systems are potentially capable of delivering end-to-end high-speed, low-power and low-resolution recognition for mobile and autonomous applications where slow, highly sophisticated and power hungry signal processing solutions are ineffective. Key aspects in the proposed approach include utilizing the spatial properties of physically embedded neural networks and propagating waves of activity therein for information processing, using dimensional collapse of imagery information into amenable TP and the use of asynchronous frames for information binding.

[1]  David Renshaw,et al.  IEEE International Symposium on Circuits and Systems (ISCAS) , 1990 .

[2]  Patrick Cavanagh,et al.  The blinking spotlight of attention , 2007, Proceedings of the National Academy of Sciences.

[3]  Peter Neri,et al.  Feature binding in zebrafish , 2012, Animal Behaviour.

[4]  Shape, orientation and size recognition of normal and ambiguous faces by a rotation and size spreading associative neural network , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[5]  Jian-Young Wu,et al.  Spiral Waves in Disinhibited Mammalian Neocortex , 2004, The Journal of Neuroscience.

[6]  Manuela Chessa,et al.  A Quantitative Comparison of Speed and Reliability for Log-Polar Mapping Techniques , 2011, ICVS.

[7]  Hananel Hazan,et al.  Stability and Topology in Reservoir Computing , 2010, MICAI.

[8]  Nouchine Hadjikhani,et al.  Migraine Aura: Retracting Particle-Like Waves in Weakly Susceptible Cortex , 2008, PloS one.

[9]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[10]  R. Gattass,et al.  Cortical visual areas in monkeys: location, topography, connections, columns, plasticity and cortical dynamics , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[11]  Massimo A. Sivilotti,et al.  Wiring considerations in analog VLSI systems, with application to field-programmable networks , 1992 .

[12]  H. C. Tuckwell,et al.  On shunting inhibition , 1986, Biological Cybernetics.

[13]  Andrew Adamatzky,et al.  Experimental reaction–diffusion pre-processor for shape recognition , 2002 .

[14]  J. Changeux,et al.  Experimental and Theoretical Approaches to Conscious Processing , 2011, Neuron.

[15]  Frank C. Hoppensteadt,et al.  Polychronous Wavefront Computations , 2009, Int. J. Bifurc. Chaos.

[16]  M. Giurfa,et al.  Configural processing enables discrimination and categorization of face-like stimuli in honeybees , 2010, Journal of Experimental Biology.

[17]  Wolfgang Maass,et al.  Liquid Computing , 2007, CiE.

[18]  James A C Ladyman Computation and Logic in the Real World , 2007 .

[19]  Jonathan Tapson,et al.  A neuromorphic cross-correlation chip , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[20]  Thomas Serre,et al.  A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex , 2005 .

[21]  S. Marinakis,et al.  Experimental and theoretical approaches , 2015 .

[22]  Reinhard Wolf,et al.  Visual pattern recognition in Drosophila involves retinotopic matching , 1993, Nature.

[23]  J.V. Arthur,et al.  Recurrently connected silicon neurons with active dendrites for one-shot learning , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[24]  S. Grossberg,et al.  From stereogram to surface: how the brain sees the world in depth. , 2009, Spatial vision.

[25]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[26]  Chiara Bartolozzi,et al.  Event-driven embodied system for feature extraction and object recognition in robotic applications , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[27]  Bernabe Linares-Barranco,et al.  Comparison between Frame-Constrained Fix-Pixel-Value and Frame-Free Spiking-Dynamic-Pixel ConvNets for Visual Processing , 2012, Front. Neurosci..

[28]  T. Poggio,et al.  Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[30]  Ramón Huerta,et al.  Fast and Robust Learning by Reinforcement Signals: Explorations in the Insect Brain , 2009, Neural Computation.

[31]  M. Tarr Visual Pattern Recognition , 1998 .

[32]  J. Fellous,et al.  The Processing of Color, Motion, and Stimulus Timing Are Anatomically Segregated in the Bumblebee Brain , 2008, The Journal of Neuroscience.

[33]  M. Dahlem,et al.  Self-induced splitting of spiral-shaped spreading depression waves in chicken retina , 1997, Experimental Brain Research.

[34]  Manoel Luis Costa,et al.  Reentrant spiral waves of spreading depression cause macular degeneration in hypoglycemic chicken retina , 2012, Proceedings of the National Academy of Sciences.

[35]  Andre van Schaik,et al.  An analogue VLSI implementation of polychromous spiking neural networks , 2011, 2011 Seventh International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[36]  Mark D. McDonnell,et al.  Input-rate modulation of gamma oscillations is sensitive to network topology, delays and short-term plasticity , 2012, Brain Research.

[37]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[38]  P. Fries Neuronal gamma-band synchronization as a fundamental process in cortical computation. , 2009, Annual review of neuroscience.

[39]  Alexandre Bernardino,et al.  A review of log-polar imaging for visual perception in robotics , 2010, Robotics and Autonomous Systems.

[40]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[41]  Thomas Serre,et al.  A Biologically Inspired System for Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[42]  Bernabé Linares-Barranco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Shih-Chii Liu,et al.  Computation with Spikes in a Winner-Take-All Network , 2009, Neural Computation.

[44]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[45]  Xiao-Jing Wang Neurophysiological and computational principles of cortical rhythms in cognition. , 2010, Physiological reviews.

[46]  Shaista Hussain,et al.  DELTRON: Neuromorphic architectures for delay based learning , 2012, 2012 IEEE Asia Pacific Conference on Circuits and Systems.

[47]  Mark D. McDonnell,et al.  The benefits of noise in neural systems: bridging theory and experiment , 2011, Nature Reviews Neuroscience.

[48]  Chrisantha Fernando,et al.  Pattern Recognition in a Bucket , 2003, ECAL.

[49]  Eric O. Postma,et al.  SCAN: A Scalable Model of Attentional Selection , 1997, Neural Networks.

[50]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  D. Schluter,et al.  Predators favour mimicry in a tropical reef fish , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[52]  D. Macmillan,et al.  Crayfish Recognize the Faces of Fight Opponents , 2008, PloS one.

[53]  Omid Kavehei,et al.  Emergence of competitive control in a memristor-based neuromorphic circuit , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[54]  Dominique Martinez,et al.  Oscillatory Synchronization Requires Precise and Balanced Feedback Inhibition in a Model of the Insect Antennal Lobe , 2005, Neural Computation.

[55]  D. Koshland Frontiers in neuroscience. , 1988, Science.

[56]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[57]  T. Sejnowski,et al.  23 problems in systems neuroscience , 2006 .

[58]  David M. Santucci,et al.  A Biologically Plausible Transform for Visual Recognition that is Invariant to Translation, Scale, and Rotation , 2011, Front. Comput. Neurosci..

[59]  Yan Meng,et al.  Modeling Activity-Dependent Plasticity in BCM Spiking Neural Networks With Application to Human Behavior Recognition , 2011, IEEE Transactions on Neural Networks.

[60]  A. Horridge Generalization in visual recognition by the honeybee (Apis mellifera): a review and explanation. , 2009, Journal of insect physiology.

[61]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[62]  Piotr Dudek,et al.  A general-purpose processor-per-pixel analog SIMD vision chip , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[63]  G. Fiorito,et al.  I Know My Neighbour: Individual Recognition in Octopus vulgaris , 2011, PloS one.

[64]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[65]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[66]  P K Ahnelt,et al.  Identification of a subtype of cone photoreceptor, likely to be blue sensitive, in the human retina , 1987, The Journal of comparative neurology.

[67]  Shoushun Chen,et al.  A bio-inspired event-based size and position invariant human posture recognition algorithm , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[68]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[69]  Martin Golubitsky,et al.  What Geometric Visual Hallucinations Tell Us about the Visual Cortex , 2002, Neural Computation.

[70]  Paolo Arena,et al.  Learning expectation in insects: A recurrent spiking neural model for spatio-temporal representation , 2012, Neural Networks.

[71]  E. Miller,et al.  Serial, Covert Shifts of Attention during Visual Search Are Reflected by the Frontal Eye Fields and Correlated with Population Oscillations , 2009, Neuron.

[72]  F. Gherardi,et al.  Revisiting social recognition systems in invertebrates , 2012, Animal Cognition.

[73]  Nuno Vasconcelos,et al.  Decision-Theoretic Saliency: Computational Principles, Biological Plausibility, and Implications for Neurophysiology and Psychophysics , 2009, Neural Computation.

[74]  Mohammad Norouzi,et al.  Stacks of convolutional Restricted Boltzmann Machines for shift-invariant feature learning , 2009, CVPR.

[75]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[76]  H. J. Reitboeck,et al.  A model for size- and rotation-invariant pattern processing in the visual system , 2004, Biological Cybernetics.

[77]  Chi-Ying Tsui,et al.  IEEE Transactions on Circuits and Systems—I:Regular Papers information for authors , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[78]  Nicolas Pinto,et al.  Why is Real-World Visual Object Recognition Hard? , 2008, PLoS Comput. Biol..

[79]  Jian-Young Wu,et al.  Propagating Waves of Activity in the Neocortex: What They Are, What They Do , 2008, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[80]  João Ranhel,et al.  Neural Assembly Computing , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[81]  Terrence J. Sejnowski,et al.  Shunting Inhibition Controls the Gain Modulation Mediated by Asynchronous Neurotransmitter Release in Early Development , 2010, PLoS Comput. Biol..

[82]  David V. Anderson,et al.  Developing large-scale field-programmable analog arrays , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[83]  H. Sompolinsky,et al.  Time-Warp–Invariant Neuronal Processing , 2009, PLoS biology.

[84]  Tobi Delbrück,et al.  A Multichip Pulse-Based Neuromorphic Infrastructure and Its Application to a Model of Orientation Selectivity , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[85]  Stephen J. Gotts,et al.  High-frequency, Long-range Coupling between Prefrontal and Visual Cortex during Sustained Attention , 2022 .

[86]  Christoph Rasche,et al.  Neuromorphic Excitable Maps for Visual Processing , 2007, IEEE Transactions on Neural Networks.

[87]  M. Land Motion and vision: why animals move their eyes , 1999, Journal of Comparative Physiology A.

[88]  Gregory Cohen,et al.  Synthesis of neural networks for spatio-temporal spike pattern recognition and processing , 2013, Front. Neurosci..

[89]  P. Dodwell Visual Pattern Recognition , 1970 .

[90]  P Cavanagh,et al.  Size and Position Invariance in the Visual System , 1978, Perception.

[91]  Bart Dierickx,et al.  Space-variant nonorthogonal structure CMOS image sensor design , 1998, IEEE J. Solid State Circuits.

[92]  G J Vachtsevanos,et al.  Gamma coherence and conscious perception , 2002, Neurology.

[93]  G. Edelman,et al.  Visual binding through reentrant connectivity and dynamic synchronization in a brain-based device. , 2004, Cerebral cortex.

[94]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[95]  Ralph Etienne-Cummings,et al.  Towards a Cortical Prosthesis: Implementing A Spike-Based HMAX Model of Visual Object Recognition in Silico , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[96]  Bertram E. Shi,et al.  Neuromorphic implementation of orientation hypercolumns , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[97]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[98]  William B. Levy,et al.  Energy Efficient Neural Codes , 1996, Neural Computation.

[99]  Gert Cauwenberghs,et al.  A Multichip Neuromorphic System for Spike-Based Visual Information Processing , 2007, Neural Computation.

[100]  Khan M. Iftekharuddin,et al.  Transformation Invariant On-Line Target Recognition , 2011, IEEE Transactions on Neural Networks.

[101]  Aurel A. Lazar,et al.  Video Time Encoding Machines , 2011, IEEE Transactions on Neural Networks.

[102]  André van Schaik,et al.  Learning the pseudoinverse solution to network weights , 2012, Neural Networks.

[103]  Bertram E. Shi,et al.  Expandable hardware for computing cortical feature maps , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[104]  H H Bülthoff,et al.  How are three-dimensional objects represented in the brain? , 1994, Cerebral cortex.

[105]  Mohammad Norouzi,et al.  Stacks of convolutional Restricted Boltzmann Machines for shift-invariant feature learning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[106]  Sebastian A. Wills,et al.  Computation with Spiking Neurons , 2004 .

[107]  Samy Bengio,et al.  Delay learning and polychronization for reservoir computing , 2008, Neurocomputing.

[108]  P. Lichtsteiner,et al.  Toward real-time particle tracking using an event-based dynamic vision sensor , 2011 .

[109]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[110]  Tetsuya Asai,et al.  Reaction-diffusion computers , 2005 .

[111]  P. N. Loxley,et al.  Ultra-fast detection of salient contours through horizontal connections in the primary visual cortex , 2011 .

[112]  M. Dahlem,et al.  A computational perspective on migraine aura , 2004, Progress in Neurobiology.

[113]  Gregory Cohen,et al.  An FPGA Implementation of a Polychronous Spiking Neural Network with Delay Adaptation , 2013, Front. Neurosci..

[114]  J. Pérez-Carrasco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate-Coding and Coincidence Processing. Application to Feed Forward ConvNets. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[115]  A. Barabasi,et al.  Network biology: understanding the cell's functional organization , 2004, Nature Reviews Genetics.

[116]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.