A Survey of Encoding Techniques for Signal Processing in Spiking Neural Networks

Biologically inspired spiking neural networks are increasingly popular in the field of artificial intelligence due to their ability to solve complex problems while being power efficient. They do so by leveraging the timing of discrete spikes as main information carrier. Though, industrial applications are still lacking, partially because the question of how to encode incoming data into discrete spike events cannot be uniformly answered. In this paper, we summarise the signal encoding schemes presented in the literature and propose a uniform nomenclature to prevent the vague usage of ambiguous definitions. Therefore we survey both, the theoretical foundations as well as applications of the encoding schemes. This work provides a foundation in spiking signal encoding and gives an overview over different application-oriented implementations which utilise the schemes.

[1]  B. Richmond,et al.  Latency: another potential code for feature binding in striate cortex. , 1996, Journal of neurophysiology.

[2]  Anthony Maida,et al.  BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticity , 2017, Neurocomputing.

[3]  Mikhail V. Kiselev Rate coding vs. temporal coding - is optimum between? , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[4]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[5]  Shih-Chii Liu,et al.  Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification , 2017, Front. Neurosci..

[6]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[7]  Subutai Ahmad,et al.  Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex , 2015, Front. Neural Circuits.

[8]  Simei Gomes Wysoski,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008, Neurocomputing.

[9]  Sander M. Bohte,et al.  Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks , 2002, IEEE Trans. Neural Networks.

[10]  Bernabé Linares-Barranco,et al.  An Event-Driven Classifier for Spiking Neural Networks Fed with Synthetic or Dynamic Vision Sensor Data , 2017, Front. Neurosci..

[11]  Andreas T. Schaefer,et al.  Maintaining Accuracy at the Expense of Speed Stimulus Similarity Defines Odor Discrimination Time in Mice , 2004, Neuron.

[12]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[13]  Simei Gomes Wysoski,et al.  Text-Independent Speaker Authentication with Spiking Neural Networks , 2007, ICANN.

[14]  E. Rolls,et al.  Information in the first spike, the order of spikes, and the number of spikes provided by neurons in the inferior temporal visual cortex , 2006, Vision Research.

[15]  J Gautrais,et al.  Rate coding versus temporal order coding: a theoretical approach. , 1998, Bio Systems.

[16]  Rasmus S Petersen,et al.  Comparison of latency and rate coding for the direction of whisker deflection in the subcortical somatosensory pathway. , 2012, Journal of neurophysiology.

[17]  Nikola Kasabov,et al.  Selection and Optimization of Temporal Spike Encoding Methods for Spiking Neural Networks , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[19]  Gregory Cohen,et al.  Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades , 2015, Front. Neurosci..

[20]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[21]  Andreas T. Schaefer,et al.  Theta oscillation coupled spike latencies yield computational vigour in a mammalian sensory system , 2003, The Journal of physiology.

[22]  Kiyoung Choi,et al.  Deep neural networks with weighted spikes , 2018, Neurocomputing.

[23]  Alois Knoll,et al.  End to End Learning of Spiking Neural Network Based on R-STDP for a Lane Keeping Vehicle , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[24]  E. Niebur,et al.  Growth patterns in the developing brain detected by using continuum mechanical tensor maps , 2022 .

[25]  Kay Chen Tan,et al.  A brain-inspired spiking neural network model with temporal encoding and learning , 2014, Neurocomputing.

[26]  Shigang Yue,et al.  Fast unsupervised learning for visual pattern recognition using spike timing dependent plasticity , 2017, Neurocomputing.

[27]  Stefano Panzeri,et al.  Correcting for the sampling bias problem in spike train information measures. , 2007, Journal of neurophysiology.

[28]  Fleur Zeldenrust,et al.  Neural Coding With Bursts—Current State and Future Perspectives , 2018, Front. Comput. Neurosci..

[29]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[30]  Vishal Sharma,et al.  A spiking neural network based on temporal encoding for electricity price time series forecasting in deregulated markets , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[31]  Jacques Gautrais,et al.  Rank order coding , 1998 .

[32]  Sungroh Yoon,et al.  Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks , 2018, 2019 56th ACM/IEEE Design Automation Conference (DAC).

[33]  Peter Blouw,et al.  Benchmarking Keyword Spotting Efficiency on Neuromorphic Hardware , 2018, NICE '19.

[34]  Benjamin Schrauwen,et al.  Compact hardware liquid state machines on FPGA for real-time speech recognition , 2008, Neural Networks.

[35]  R. Galamboš,et al.  THE RESPONSE OF SINGLE AUDITORY-NERVE FIBERS TO ACOUSTIC STIMULATION , 1943 .

[36]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[37]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[38]  Christian K. Machens,et al.  Efficient codes and balanced networks , 2016, Nature Neuroscience.

[39]  Arnaud Delorme,et al.  Face identification using one spike per neuron: resistance to image degradations , 2001, Neural Networks.

[40]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[41]  Walter Senn,et al.  Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics , 2007, Neural Computation.

[42]  J. Rouat,et al.  Exploration of rank order coding with spiking neural networks for speech recognition , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[43]  Wolfgang Maass,et al.  On the Computational Complexity of Networks of Spiking Neurons , 1994, NIPS.

[44]  Tobi Delbrück,et al.  Event-based 64-channel binaural silicon cochlea with Q enhancement mechanisms , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[45]  Sethu Vijayakumar,et al.  Information about Complex Fingertip Parameters in Individual Human Tactile Afferent Neurons , 2009, The Journal of Neuroscience.

[46]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[47]  Giacomo Indiveri,et al.  Obstacle Avoidance and Target Acquisition for Robot Navigation Using a Mixed Signal Analog/Digital Neuromorphic Processing System , 2017, Front. Neurorobot..

[48]  Meng Li,et al.  Neural Code—Neural Self-information Theory on How Cell-Assembly Code Rises from Spike Time and Neuronal Variability , 2017, Front. Cell. Neurosci..

[49]  R. Johansson,et al.  First spikes in ensembles of human tactile afferents code complex spatial fingertip events , 2004, Nature Neuroscience.

[50]  Timothée Masquelier,et al.  STDP-based spiking deep neural networks for object recognition , 2016, Neural Networks.

[51]  Shengyuan Zhou,et al.  TDSNN: From Deep Neural Networks to Deep Spike Neural Networks with Temporal-Coding , 2019, AAAI.

[52]  Nikola K. Kasabov,et al.  Spike-time encoding as a data compression technique for pattern recognition of temporal data , 2017, Inf. Sci..

[53]  W. Singer,et al.  Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[54]  Subutai Ahmad,et al.  Unsupervised real-time anomaly detection for streaming data , 2017, Neurocomputing.

[55]  N. Logothetis,et al.  Phase-of-Firing Coding of Natural Visual Stimuli in Primary Visual Cortex , 2008, Current Biology.

[56]  Bernabé Linares-Barranco,et al.  Poker-DVS and MNIST-DVS. Their History, How They Were Made, and Other Details , 2015, Front. Neurosci..

[57]  Simon J. Thorpe,et al.  Coding static natural images using spiking event times: do neurons Cooperate? , 2004, IEEE Transactions on Neural Networks.

[58]  Anthony S. Maida,et al.  Multi-layer unsupervised learning in a spiking convolutional neural network , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[59]  Pierre Kornprobst,et al.  Rank Order Coding: a Retinal Information Decoding Strategy Revealed by Large-Scale Multielectrode Array Retinal Recordings , 2016, eNeuro.

[60]  A. Sboev,et al.  Solving a classification task by spiking neural network with STDP based on rate and temporal input encoding , 2020, Mathematical Methods in the Applied Sciences.

[61]  Tobi Delbrück,et al.  DHP19: Dynamic Vision Sensor 3D Human Pose Dataset , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[62]  Maryam Gholami Doborjeh,et al.  Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications , 2016, Neural Networks.

[63]  Tobi Delbrück,et al.  Fast sensory motor control based on event-based hybrid neuromorphic-procedural system , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[64]  Arnaud Delorme,et al.  Networks of integrate-and-fire neurons using Rank Order Coding B: Spike timing dependent plasticity and emergence of orientation selectivity , 2001, Neurocomputing.

[65]  Toshimichi Saito,et al.  Quantized Spiking Neuron With A/D Conversion Functions , 2006, IEEE Transactions on Circuits and Systems II: Express Briefs.

[66]  S. Thorpe,et al.  Spike times make sense , 2005, Trends in Neurosciences.

[67]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[68]  Qinru Qiu,et al.  Real-time anomaly detection for streaming data using burst code on a neurosynaptic processor , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[69]  Timothée Masquelier,et al.  Deep Learning in Spiking Neural Networks , 2018, Neural Networks.

[70]  E. Adrian,et al.  The impulses produced by sensory nerve endings , 1926, The Journal of physiology.

[71]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  B. Doiron,et al.  Interval coding. I. Burst interspike intervals as indicators of stimulus intensity. , 2007, Journal of neurophysiology.

[73]  Tim Gollisch,et al.  Rapid Neural Coding in the Retina with Relative Spike Latencies , 2008, Science.

[74]  Arnaud Delorme,et al.  Spike-based strategies for rapid processing , 2001, Neural Networks.

[75]  Emilio Kropff,et al.  Place cells, grid cells, and the brain's spatial representation system. , 2008, Annual review of neuroscience.

[76]  Kaushik Roy,et al.  Unsupervised regenerative learning of hierarchical features in Spiking Deep Networks for object recognition , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[77]  Simei Gomes Wysoski,et al.  Adaptive Spiking Neural Networks for Audiovisual Pattern Recognition , 2007, ICONIP.

[78]  Shih-Chii Liu,et al.  Effective sensor fusion with event-based sensors and deep network architectures , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[79]  Filip Ponulak,et al.  Introduction to spiking neural networks: Information processing, learning and applications. , 2011, Acta neurobiologiae experimentalis.

[80]  Bernabé Linares-Barranco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Lon Turnbull,et al.  The string method of burst identification in neuronal spike trains , 2005, Journal of Neuroscience Methods.

[82]  Si Wu,et al.  Population Coding and Decoding in a Neural Field: A Computational Study , 2002, Neural Computation.

[83]  Amine Bermak,et al.  Spike Latency Coding in Biologically Inspired Microelectronic Nose , 2011, IEEE Transactions on Biomedical Circuits and Systems.

[84]  Nikola Kasabov,et al.  Framework for Knowledge Driven Optimisation Based Data Encoding for Brain Data Modelling Using Spiking Neural Network Architecture , 2015 .

[85]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[86]  Shih-Chii Liu,et al.  Conversion of analog to spiking neural networks using sparse temporal coding , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[87]  Alois Knoll,et al.  A Survey of Robotics Control Based on Learning-Inspired Spiking Neural Networks , 2018, Front. Neurorobot..

[88]  Nikola Kasabov,et al.  Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence , 2018, Springer Series on Bio- and Neurosystems.

[89]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[90]  J. J. Hopfield,et al.  Pattern recognition computation using action potential timing for stimulus representation , 1995, Nature.

[91]  Marcelo A. Montemurro,et al.  Spike-Phase Coding Boosts and Stabilizes Information Carried by Spatial and Temporal Spike Patterns , 2009, Neuron.

[92]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[93]  Alexander Borst,et al.  Information theory and neural coding , 1999, Nature Neuroscience.

[94]  Adrienne L. Fairhall,et al.  Efficiency and ambiguity in an adaptive neural code , 2001, Nature.

[95]  Nikola K. Kasabov,et al.  A Retinotopic Spiking Neural Network System for Accurate Recognition of Moving Objects Using NeuCube and Dynamic Vision Sensors , 2018, Front. Comput. Neurosci..

[96]  Nikola K. Kasabov,et al.  EEG Classification with BSA Spike Encoding Algorithm and Evolving Probabilistic Spiking Neural Network , 2011, ICONIP.

[97]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[98]  Haizhou Li,et al.  Rapid Feedforward Computation by Temporal Encoding and Learning With Spiking Neurons , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[99]  B. Schrauwen,et al.  BSA, a fast and accurate spike train encoding scheme , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[100]  Kyeong-Sik Min,et al.  Spatial-Pooling Memristor Crossbar Converting Sensory Information to Sparse Distributed Representation of Cortical Neurons , 2018, IEEE Transactions on Nanotechnology.