Spiking Neural Networks Hardware Implementations and Challenges

Neuromorphic computing is henceforth a major research field for both academic and industrial actors. As opposed to Von Neumann machines, brain-inspired processors aim at bringing closer the memory and the computational elements to efficiently evaluate machine learning algorithms. Recently, spiking neural networks, a generation of cognitive algorithms employing computational primitives mimicking neuron and synapse operational principles, have become an important part of deep learning. They are expected to improve the computational performance and efficiency of neural networks, but they are best suited for hardware able to support their temporal dynamics. In this survey, we present the state of the art of hardware implementations of spiking neural networks and the current trends in algorithm elaboration from model selection to training mechanisms. The scope of existing solutions is extensive; we thus present the general framework and study on a case-by-case basis the relevant particularities. We describe the strategies employed to leverage the characteristics of these event-driven algorithms at the hardware level and discuss their related advantages and challenges.

[1]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[2]  Shih-Chii Liu,et al.  A Neuromorphic VLSI Circuit for Spike-Based Random Sampling , 2018, IEEE Transactions on Emerging Topics in Computing.

[3]  Timothée Masquelier,et al.  Bio-inspired unsupervised learning of visual features leads to robust invariant object recognition , 2015, Neurocomputing.

[4]  Jim Harkin,et al.  Homeostatic Fault Tolerance in Spiking Neural Networks: A Dynamic Hardware Perspective , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  Tobi Delbruck,et al.  A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision Sensor , 2014, IEEE Journal of Solid-State Circuits.

[6]  Yu Cao,et al.  Monolithic 3D IC designs for low-power deep neural networks targeting speech recognition , 2017, 2017 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED).

[7]  Benjamin Schrauwen,et al.  Compact hardware liquid state machines on FPGA for real-time speech recognition , 2008, Neural Networks.

[8]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[9]  Francky Catthoor,et al.  Mapping of local and global synapses on spiking neuromorphic hardware , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[10]  Andrew M. Tyrrell,et al.  Challenges of evolvable hardware: past, present and the path to a promising future , 2011, Genetic Programming and Evolvable Machines.

[11]  Jim Harkin,et al.  SPANNER: A Self-Repairing Spiking Neural Network Hardware Architecture , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Gopalakrishnan Srinivasan,et al.  Training Deep Spiking Convolutional Neural Networks With STDP-Based Unsupervised Pre-training Followed by Supervised Fine-Tuning , 2018, Front. Neurosci..

[13]  E. Vianello,et al.  Spiking Neural Networks Based on OxRAM Synapses for Real-Time Unsupervised Spike Sorting , 2016, Frontiers in neuroscience.

[14]  Yoshua Bengio,et al.  An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks , 2013, ICLR.

[15]  P. Narayanan,et al.  Recent progress in analog memory-based accelerators for deep learning , 2018, Journal of Physics D: Applied Physics.

[16]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[17]  Olivier Temam,et al.  Capacitance of TSVs in 3-D stacked chips a problem? Not for neuromorphic systems! , 2012, DAC Design Automation Conference 2012.

[18]  Gregory K. Chen,et al.  A 4096-Neuron 1M-Synapse 3.8PJ/SOP Spiking Neural Network with On-Chip STDP Learning and Sparse Weights in 10NM FinFET CMOS , 2018, 2018 IEEE Symposium on VLSI Circuits.

[19]  A. Holden Competition and cooperation in neural nets , 1983 .

[20]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[21]  Andrew S. Cassidy,et al.  Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[22]  Yoshua Bengio,et al.  Measuring the tendency of CNNs to Learn Surface Statistical Regularities , 2017, ArXiv.

[23]  Kaushik Roy,et al.  Cross-Layer Design Exploration for Energy-Quality Tradeoffs in Spiking and Non-Spiking Deep Artificial Neural Networks , 2018, IEEE Transactions on Multi-Scale Computing Systems.

[24]  John W. Backus,et al.  Can programming be liberated from the von Neumann style?: a functional style and its algebra of programs , 1978, CACM.

[25]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[26]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[27]  G. Indiveri,et al.  Neuromorphic architectures for spiking deep neural networks , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[28]  Peng Li,et al.  Online Adaptation and Energy Minimization for Hardware Recurrent Spiking Neural Networks , 2018, ACM J. Emerg. Technol. Comput. Syst..

[29]  Anand Raghunathan,et al.  Approximate computing for spiking neural networks , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[30]  Daniel Matolin,et al.  A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS , 2011, IEEE Journal of Solid-State Circuits.

[31]  Indar Sugiarto,et al.  Optimized task graph mapping on a many-core neuromorphic supercomputer , 2017, 2017 IEEE High Performance Extreme Computing Conference (HPEC).

[32]  Zhengya Zhang,et al.  A Sparse Coding Neural Network ASIC With On-Chip Learning for Feature Extraction and Encoding , 2015, IEEE Journal of Solid-State Circuits.

[33]  Terrence J. Sejnowski,et al.  Gradient Descent for Spiking Neural Networks , 2017, NeurIPS.

[34]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[35]  Stefan Schliebs,et al.  Span: Spike Pattern Association Neuron for Learning Spatio-Temporal Spike Patterns , 2012, Int. J. Neural Syst..

[36]  David Kappel,et al.  Deep Rewiring: Training very sparse deep networks , 2017, ICLR.

[37]  Gregory Cohen,et al.  Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades , 2015, Front. Neurosci..

[38]  Wenrui Zhang,et al.  Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks , 2018, NeurIPS.

[39]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[40]  F. Ponulak ReSuMe-New Supervised Learning Method for Spiking Neural Networks , 2005 .

[41]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[42]  C D Salzman,et al.  Neural mechanisms for forming a perceptual decision. , 1994, Science.

[43]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[44]  Kaushik Roy,et al.  Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning , 2016, Scientific Reports.

[45]  Siddharth Joshi,et al.  Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines , 2015, Front. Neurosci..

[46]  C. Pittenger,et al.  Stress, Depression, and Neuroplasticity: A Convergence of Mechanisms , 2008, Neuropsychopharmacology.

[47]  Zhengya Zhang,et al.  A 127mW 1.63TOPS sparse spatio-temporal cognitive SoC for action classification and motion tracking in videos , 2017, 2017 Symposium on VLSI Circuits.

[48]  Kaushik Roy,et al.  STDP-Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy-Efficient Recognition , 2017, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[49]  Andrew S. Cassidy,et al.  Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[50]  Bernabé Linares-Barranco,et al.  A Configurable Event-Driven Convolutional Node with Rate Saturation Mechanism for Modular ConvNet Systems Implementation , 2018, Front. Neurosci..

[51]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[52]  Enrico Macii,et al.  Optimizing Network Traffic for Spiking Neural Network Simulations on Densely Interconnected Many-Core Neuromorphic Platforms , 2018, IEEE Transactions on Emerging Topics in Computing.

[53]  Ning Ma,et al.  Designing bio-inspired autonomous error-tolerant massively parallel computing architectures , 2017, 2017 30th IEEE International System-on-Chip Conference (SOCC).

[54]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[55]  Peng Li,et al.  AP-STDP: A novel self-organizing mechanism for efficient reservoir computing , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[56]  V. Canals,et al.  A Stochastic Spiking Neural Network for Virtual Screening , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[57]  David Bol,et al.  Impact of the AER-induced timing distortion on Spiking Neural Networks implementing DSP , 2016, 2016 12th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME).

[58]  Giacomo Indiveri,et al.  Analog circuits for mixed-signal neuromorphic computing architectures in 28 nm FD-SOI technology , 2017, 2017 IEEE SOI-3D-Subthreshold Microelectronics Technology Unified Conference (S3S).

[59]  Gert Cauwenberghs,et al.  Fast classification using sparsely active spiking networks , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[60]  Mark Horowitz,et al.  1.1 Computing's energy problem (and what we can do about it) , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[61]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[62]  Ángel Rodríguez-Vázquez,et al.  Applications of event‐based image sensors—Review and analysis , 2018, Int. J. Circuit Theory Appl..

[63]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[64]  Arnab Roy,et al.  A Programmable Event-driven Architecture for Evaluating Spiking Neural Networks , 2017, 2017 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED).

[65]  Sungho Kim,et al.  Impact of Synaptic Device Variations on Pattern Recognition Accuracy in a Hardware Neural Network , 2018, Scientific Reports.

[66]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[67]  Gert Cauwenberghs,et al.  Synaptic sampling in hardware spiking neural networks , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[68]  Xiaochen Peng,et al.  XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[69]  Timothée Masquelier,et al.  First-Spike-Based Visual Categorization Using Reward-Modulated STDP , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[70]  Tobi Delbrück,et al.  DDD17: End-To-End DAVIS Driving Dataset , 2017, ArXiv.

[71]  S. Herculano‐Houzel Scaling of Brain Metabolism with a Fixed Energy Budget per Neuron: Implications for Neuronal Activity, Plasticity and Evolution , 2011, PloS one.

[72]  F. Grassia,et al.  Spike pattern recognition using artificial neuron and spike-timing-dependent plasticity implemented on a multi-core embedded platform , 2018, Artificial Life and Robotics.

[73]  D. Querlioz,et al.  Visual Pattern Extraction Using Energy-Efficient “2-PCM Synapse” Neuromorphic Architecture , 2012, IEEE Transactions on Electron Devices.

[74]  Jie Han,et al.  Approximate computing: An emerging paradigm for energy-efficient design , 2013, 2013 18th IEEE European Test Symposium (ETS).

[75]  Y. Leblebici,et al.  Large-scale neural networks implemented with non-volatile memory as the synaptic weight element: Comparative performance analysis (accuracy, speed, and power) , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[76]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[77]  Tao Liu,et al.  MT-spike: A multilayer time-based spiking neuromorphic architecture with temporal error backpropagation , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[78]  Alberto Mazzoni,et al.  Neuromorphic Artificial Touch for Categorization of Naturalistic Textures , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[79]  Claus Nebauer,et al.  Evaluation of convolutional neural networks for visual recognition , 1998, IEEE Trans. Neural Networks.

[80]  Gu-Yeon Wei,et al.  14.3 A 28nm SoC with a 1.2GHz 568nJ/prediction sparse deep-neural-network engine with >0.1 timing error rate tolerance for IoT applications , 2017, 2017 IEEE International Solid-State Circuits Conference (ISSCC).

[81]  Peng Li,et al.  Performance and robustness of bio-inspired digital liquid state machines: A case study of speech recognition , 2017, Neurocomputing.

[82]  Olivier Bichler,et al.  Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction , 2011, 2011 International Electron Devices Meeting.

[83]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[84]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[85]  J. Yang,et al.  Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. , 2017, Nature materials.

[86]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[87]  Yong Zhang,et al.  A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing , 2015, ACM J. Emerg. Technol. Comput. Syst..

[88]  Ryad Benosman,et al.  HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[89]  R. French Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[90]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[91]  Arindam Basu,et al.  Low-Power, Adaptive Neuromorphic Systems: Recent Progress and Future Directions , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[92]  Eby G. Friedman,et al.  Three-dimensional Integrated Circuit Design , 2008 .

[93]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[94]  Peng Li,et al.  SSO-LSM: A Sparse and Self-Organizing architecture for Liquid State Machine based neural processors , 2016, 2016 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[95]  Zhengya Zhang,et al.  A Native Stochastic Computing Architecture Enabled by Memristors , 2014, IEEE Transactions on Nanotechnology.

[96]  D. Feldman Synaptic mechanisms for plasticity in neocortex. , 2009, Annual review of neuroscience.

[97]  Simon J. Thorpe,et al.  Combining STDP and Reward-Modulated STDP in Deep Convolutional Spiking Neural Networks for Digit Recognition , 2018, ArXiv.

[98]  Kaushik Roy,et al.  RESPARC: A reconfigurable and energy-efficient architecture with Memristive Crossbars for deep Spiking Neural Networks , 2017, 2017 54th ACM/EDAC/IEEE Design Automation Conference (DAC).

[99]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[100]  Marian Verhelst,et al.  A 0.3–2.6 TOPS/W precision-scalable processor for real-time large-scale ConvNets , 2016, 2016 IEEE Symposium on VLSI Circuits (VLSI-Circuits).

[101]  Steve B. Furber,et al.  A hierachical configuration system for a massively parallel neural hardware platform , 2012, CF '12.

[102]  Jim D. Garside,et al.  Dynamic voltage and frequency scaling for neuromorphic many-core systems , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[103]  Dharmendra S. Modha,et al.  Backpropagation for Energy-Efficient Neuromorphic Computing , 2015, NIPS.

[104]  Shih-Chii Liu,et al.  Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification , 2017, Front. Neurosci..

[105]  O. Faynot,et al.  3DVLSI with CoolCube process: An alternative path to scaling , 2015, 2015 Symposium on VLSI Technology (VLSI Technology).

[106]  Ojas Parekh,et al.  Energy Scaling Advantages of Resistive Memory Crossbar Based Computation and Its Application to Sparse Coding , 2016, Front. Neurosci..

[107]  E. Vianello,et al.  HfO2-Based OxRAM Devices as Synapses for Convolutional Neural Networks , 2015, IEEE Transactions on Electron Devices.

[108]  Surya Ganguli,et al.  SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks , 2017, Neural Computation.

[109]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[110]  L’oubli catastrophique it,et al.  Avoiding catastrophic forgetting by coupling two reverberating neural networks , 2004 .

[111]  Zoha Pajouhi Energy efficient neuromorphic processing using spintronic memristive device with dedicated synaptic and neuron terminology , 2018, 2018 19th International Symposium on Quality Electronic Design (ISQED).

[112]  Ran El-Yaniv,et al.  Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations , 2016, J. Mach. Learn. Res..

[113]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[114]  J. Barry Richmond,et al.  Neural Coding , 2014, Encyclopedia of Computational Neuroscience.

[115]  R MahapatraNihar,et al.  The processor-memory bottleneck , 1999 .

[116]  Deepak Khosla,et al.  Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition , 2014, International Journal of Computer Vision.

[117]  S. Datta,et al.  Ultra-low power probabilistic IMT neurons for stochastic sampling machines , 2017, 2017 Symposium on VLSI Circuits.

[118]  Saibal Mukhopadhyay,et al.  3-D Stacked Image Sensor With Deep Neural Network Computation , 2018, IEEE Sensors Journal.

[119]  R. Jordan,et al.  NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[120]  E. Vianello,et al.  Bio-Inspired Stochastic Computing Using Binary CBRAM Synapses , 2013, IEEE Transactions on Electron Devices.

[121]  Garrick Orchard,et al.  Hybrid Neural Network, An Efficient Low-Power Digital Hardware Implementation of Event-based Artificial Neural Network , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[122]  Kay Chen Tan,et al.  Sparse Temporal Encoding of Visual Features for Robust Object Recognition by Spiking Neurons , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[123]  Qian Wang,et al.  General-purpose LSM learning processor architecture and theoretically guided design space exploration , 2015, 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[124]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[125]  Conrad D. James,et al.  Impact of Linearity and Write Noise of Analog Resistive Memory Devices in a Neural Algorithm Accelerator , 2017, 2017 IEEE International Conference on Rebooting Computing (ICRC).

[126]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[127]  Pinaki Mazumder,et al.  Online Supervised Learning for Hardware-Based Multilayer Spiking Neural Networks Through the Modulation of Weight-Dependent Spike-Timing-Dependent Plasticity , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[128]  Katsuyuki Sakuma,et al.  3D Integration in VLSI Circuits Implementation Technologies and Applications , 2019 .

[129]  Vijay Kumar,et al.  The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception , 2018, IEEE Robotics and Automation Letters.

[130]  Giacomo Indiveri,et al.  Memory and Information Processing in Neuromorphic Systems , 2015, Proceedings of the IEEE.

[131]  Yong Zhang,et al.  An energy efficient approximate adder with carry skip for error resilient neuromorphic VLSI systems , 2013, 2013 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[132]  Zhen Zhou,et al.  Neuromorphic 3D Integrated Circuit: A Hybrid, Reliable and Energy Efficient Approach for Next Generation Computing , 2017, ACM Great Lakes Symposium on VLSI.

[133]  Igor Carron,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016 .

[134]  Razvan V. Florian,et al.  The Chronotron: A Neuron That Learns to Fire Temporally Precise Spike Patterns , 2010, PloS one.

[135]  Dongsuk Jeon,et al.  7.6 A 65nm 236.5nJ/Classification Neuromorphic Processor with 7.5% Energy Overhead On-Chip Learning Using Direct Spike-Only Feedback , 2019, 2019 IEEE International Solid- State Circuits Conference - (ISSCC).

[136]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[137]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[138]  Haizhou Li,et al.  Spike Timing or Rate? Neurons Learn to Make Decisions for Both Through Threshold-Driven Plasticity , 2019, IEEE Transactions on Cybernetics.

[139]  Manish Kumar Large-scale neuromorphic computing systems , 2016 .

[140]  Sander M. Bohte,et al.  Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model , 2012, NIPS.

[141]  Takao Marukame,et al.  Proposal, analysis and demonstration of Analog/Digital-mixed Neural Networks based on memristive device arrays , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[142]  Sudhakar Yalamanchili,et al.  Neurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[143]  Bernabé Linares-Barranco,et al.  Hardware implementation of convolutional STDP for on-line visual feature learning , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[144]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[145]  Pinaki Mazumder,et al.  A Low-Power Hardware Architecture for On-Line Supervised Learning in Multi-Layer Spiking Neural Networks , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[146]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[147]  Sander M. Bohte,et al.  Fast and Efficient Asynchronous Neural Computation with Adapting Spiking Neural Networks , 2016, ArXiv.

[148]  Yiran Chen,et al.  Low-power neuromorphic speech recognition engine with coarse-grain sparsity , 2017, 2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC).

[149]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[150]  Zhengya Zhang,et al.  A 640M pixel/s 3.65mW sparse event-driven neuromorphic object recognition processor with on-chip learning , 2015, 2015 Symposium on VLSI Circuits (VLSI Circuits).

[151]  Lei Deng,et al.  Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks , 2017, Front. Neurosci..

[152]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[153]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[154]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[155]  Gert Cauwenberghs,et al.  Hierarchical Address Event Routing for Reconfigurable Large-Scale Neuromorphic Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[156]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[157]  A. Calderon,et al.  Handwritten Digit Recognition using Convolutional Neural Networks and Gabor filters , 2003 .

[158]  Marie Bernert,et al.  Fully unsupervised online spike sorting based on an artificial spiking neural network , 2017 .

[159]  Yiran Chen,et al.  Neu-NoC: A high-efficient interconnection network for accelerated neuromorphic systems , 2018, 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC).

[160]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[161]  Wolf Singer,et al.  Neuronal Synchrony: A Versatile Code for the Definition of Relations? , 1999, Neuron.

[162]  Bipin Rajendran,et al.  Spiking neural networks for handwritten digit recognition - Supervised learning and network optimization , 2018, Neural Networks.

[163]  Yu Wang,et al.  Binary convolutional neural network on RRAM , 2017, 2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC).

[164]  Razvan V. Florian,et al.  Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity , 2007, Neural Computation.

[165]  Ali Farhadi,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016, ECCV.

[166]  Gianluca Tempesti,et al.  Fault Tolerance Using Dynamic Reconfiguration on the POEtic Tissue , 2007, IEEE Transactions on Evolutionary Computation.

[167]  Jiale Liang,et al.  Cross-Point Memory Array Without Cell Selectors—Device Characteristics and Data Storage Pattern Dependencies , 2010, IEEE Transactions on Electron Devices.

[168]  Steve B. Furber,et al.  A fixed point exponential function accelerator for a neuromorphic many-core system , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[169]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[170]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[171]  Adrienne L. Fairhall,et al.  Efficiency and ambiguity in an adaptive neural code , 2001, Nature.

[172]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[173]  Bernabé Linares-Barranco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[174]  Hesham Mostafa,et al.  Supervised Learning Based on Temporal Coding in Spiking Neural Networks , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[175]  André van Schaik,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[176]  Sung Kyu Lim,et al.  Design and Architectural Co-optimization of Monolithic 3D Liquid State Machine-based Neuromorphic Processor , 2018, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[177]  R. Douglas,et al.  Event-Based Neuromorphic Systems , 2015 .

[178]  Gregory Cohen,et al.  Synthesis of neural networks for spatio-temporal spike pattern recognition and processing , 2013, Front. Neurosci..

[179]  Bin Deng,et al.  Real-Time Neuromorphic System for Large-Scale Conductance-Based Spiking Neural Networks , 2019, IEEE Transactions on Cybernetics.

[180]  Tobi Delbrück,et al.  Steering a predator robot using a mixed frame/event-driven convolutional neural network , 2016, 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP).

[181]  Matthew J. Marinella,et al.  Ziksa: On-chip learning accelerator with memristor crossbars for multilevel neural networks , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[182]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[183]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[184]  Andreas Mayr,et al.  CrossNets: High‐Performance Neuromorphic Architectures for CMOL Circuits , 2003, Annals of the New York Academy of Sciences.

[185]  Olivier Temam,et al.  The improbable but highly appropriate marriage of 3D stacking and neuromorphic accelerators , 2014, 2014 International Conference on Compilers, Architecture and Synthesis for Embedded Systems (CASES).

[186]  Nor Hisham Hamid,et al.  Bio-inspired fault tolerant network on chip , 2017, Integr..

[187]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[188]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[189]  S. Thorpe,et al.  STDP-based spiking deep convolutional neural networks for object recognition , 2018 .

[190]  Kang L. Wang,et al.  Low-power non-volatile spintronic memory: STT-RAM and beyond , 2013 .

[191]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[192]  Zhengkun Yi,et al.  Recognizing tactile surface roughness with a biomimetic fingertip: A soft neuromorphic approach , 2017, Neurocomputing.

[193]  Johannes Schemmel,et al.  An Accelerated LIF Neuronal Network Array for a Large-Scale Mixed-Signal Neuromorphic Architecture , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[194]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[195]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[196]  A. Aertsen,et al.  Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding , 2010, Nature Reviews Neuroscience.

[197]  Byung-Gook Park,et al.  Spiking Neural Network Using Synaptic Transistors and Neuron Circuits for Pattern Recognition With Noisy Images , 2018, IEEE Electron Device Letters.

[198]  BeigneEdith,et al.  Spiking Neural Networks Hardware Implementations and Challenges , 2019 .

[199]  Gang Pan,et al.  Darwin: a neuromorphic hardware co-processor based on Spiking Neural Networks , 2015, Science China Information Sciences.

[200]  M. Pelcat,et al.  Tactics to Directly Map CNN Graphs on Embedded FPGAs , 2017, IEEE Embedded Systems Letters.

[201]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[202]  Antoine Dupret,et al.  Event-Based, Timescale Invariant Unsupervised Online Deep Learning With STDP , 2018, Front. Comput. Neurosci..

[203]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[204]  Dario Floreano,et al.  Hardware spiking neural network with run-time reconfigurable connectivity in an autonomous robot , 2003, NASA/DoD Conference on Evolvable Hardware, 2003. Proceedings..

[205]  Chaitali Chakrabarti,et al.  Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations , 2017, 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[206]  Shih-Chii Liu,et al.  Conversion of analog to spiking neural networks using sparse temporal coding , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[207]  Huajin Tang,et al.  Precise-Spike-Driven Synaptic Plasticity: Learning Hetero-Association of Spatiotemporal Spike Patterns , 2013, PloS one.

[208]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[209]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[210]  Giacomo Indiveri,et al.  A low-power adaptive integrate-and-fire neuron circuit , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..