Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network

This paper introduces an event-driven feedforward categorization system, which takes data from a temporal contrast address event representation (AER) sensor. The proposed system extracts bio-inspired cortex-like features and discriminates different patterns using an AER based tempotron classifier (a network of leaky integrate-and-fire spiking neurons). One of the system's most appealing characteristics is its event-driven processing, with both input and features taking the form of address events (spikes). The system was evaluated on an AER posture dataset and compared with two recently developed bio-inspired models. Experimental results have shown that it consumes much less simulation time while still maintaining comparable performance. In addition, experiments on the Mixed National Institute of Standards and Technology (MNIST) image dataset have demonstrated that the proposed system can work not only on raw AER data but also on images (with a preprocessing step to convert images into AER events) and that it can maintain competitive accuracy even when noise is added. The system was further evaluated on the MNIST dynamic vision sensor dataset (in which data is recorded using an AER dynamic vision sensor), with testing accuracy of 88.14%.

[1]  Thomas Serre,et al.  Realistic Modeling of Simple and Complex Cell Tuning in the HMAX Model, and Implications for Invariant Object Recognition in Cortex , 2004 .

[2]  Shoushun Chen,et al.  A 64×64 pixels UWB wireless temporal-difference digital image sensor , 2010, ISCAS.

[3]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Tomaso Poggio,et al.  Learning a dictionary of shape-components in visual cortex: comparison with neurons, humans and machines , 2006 .

[5]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[6]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[7]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[8]  Bernabé Linares-Barranco,et al.  On Real-Time AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing , 2008, IEEE Transactions on Neural Networks.

[9]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[10]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[11]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[12]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[13]  Bo Zhao,et al.  A 64$\,\times\,$64 CMOS Image Sensor With On-Chip Moving Object Detection and Localization , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Haizhou Li,et al.  A Spike-Timing-Based Integrated Model for Pattern Recognition , 2013, Neural Computation.

[15]  Shoushun Chen,et al.  A 64 $\times$ 64 Pixels UWB Wireless Temporal-Difference Digital Image Sensor , 2012, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[16]  Timothée Masquelier,et al.  Competitive STDP-Based Spike Pattern Learning , 2009, Neural Computation.

[17]  Marco Weiss,et al.  Open-source Projects , 2007 .

[18]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[19]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[20]  Thomas Serre,et al.  A Biologically Inspired System for Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Tobi Delbrück,et al.  Fast sensory motor control based on event-based hybrid neuromorphic-procedural system , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[23]  Bernabé Linares-Barranco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  André van Schaik,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[25]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.

[26]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[27]  Andrew D. Back,et al.  Radial Basis Functions , 2001 .

[28]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[29]  Shih-Chii Liu,et al.  Neuromorphic sensory systems , 2010, Current Opinion in Neurobiology.

[30]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[31]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[32]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[33]  Shoushun Chen,et al.  Live demonstration: A high-speed-pass asynchronous motion detection sensor , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[34]  Bo Zhao,et al.  Realtime feature extraction using MAX-like convolutional network for human posture recognition , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[35]  Haizhou Li,et al.  Rapid Feedforward Computation by Temporal Encoding and Learning With Spiking Neurons , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[36]  A.N. Belbachir,et al.  Embedded Vision System for Real-Time Object Tracking using an Asynchronous Transient Vision Sensor , 2006, 2006 IEEE 12th Digital Signal Processing Workshop & 4th IEEE Signal Processing Education Workshop.

[37]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[38]  Gert Cauwenberghs,et al.  A Multichip Neuromorphic System for Spike-Based Visual Information Processing , 2007, Neural Computation.

[39]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[40]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[41]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[42]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[43]  Eugenio Culurciello,et al.  Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Craig T. Jin,et al.  An Active 2-D Silicon Cochlea , 2008, IEEE Transactions on Biomedical Circuits and Systems.

[45]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[46]  S. Thorpe,et al.  Seeking Categories in the Brain , 2001, Science.

[47]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[48]  Tobi Delbrück,et al.  A Multichip Pulse-Based Neuromorphic Infrastructure and Its Application to a Model of Orientation Selectivity , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[49]  Bernabé Linares-Barranco,et al.  A 128$\,\times$ 128 1.5% Contrast Sensitivity 0.9% FPN 3 µs Latency 4 mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers , 2013, IEEE Journal of Solid-State Circuits.

[50]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.