Hybrid Neural Network, An Efficient Low-Power Digital Hardware Implementation of Event-based Artificial Neural Network

Interest in event-based vision sensors has proliferated in recent years, with innovative technology becoming more accessible to new researchers and highlighting such sensors' potential to enable low-latency sensing at low computational cost. These sensors can outperform frame-based vision sensors regarding data compression, dynamic range, temporal resolution and power efficiency. However, available mature frame-based processing methods by using Artificial Neural Networks (ANNs) surpass Spiking Neural Networks (SNNs) in terms of accuracy of recognition. In this paper, we introduce a Hybrid Neural Network which is an intermediate solution to exploit advantages of both event-based and frame-based processing. We have implemented this network in FPGA and benchmarked its performance by using different event-based versions of MNIST dataset. HDL codes for this project are available for academic purpose upon request.

[1]  B. McNaughton,et al.  Packet-based communication in the cortex , 2015, Nature Reviews Neuroscience.

[2]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[3]  Tobi Delbrück,et al.  Retinomorphic Event-Based Vision Sensors: Bioinspired Cameras With Spiking Output , 2014, Proceedings of the IEEE.

[4]  Sungho Kim,et al.  4.1 A 640×480 dynamic vision sensor with a 9µm pixel and 300Meps address-event representation , 2017, 2017 IEEE International Solid-State Circuits Conference (ISSCC).

[5]  Gregory Cohen,et al.  Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades , 2015, Front. Neurosci..

[6]  Bernabé Linares-Barranco,et al.  Hardware implementation of convolutional STDP for on-line visual feature learning , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[7]  Bernabé Linares-Barranco,et al.  On Multiple AER Handshaking Channels Over High-Speed Bit-Serial Bidirectional LVDS Links With Flow-Control and Clock-Correction on Commercial FPGAs for Scalable Neuromorphic Systems , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[8]  Bernabé Linares-Barranco,et al.  A 128$\,\times$ 128 1.5% Contrast Sensitivity 0.9% FPN 3 µs Latency 4 mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers , 2013, IEEE Journal of Solid-State Circuits.

[9]  Bernabé Linares-Barranco,et al.  A 3.6 $\mu$ s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor , 2011, IEEE Journal of Solid-State Circuits.

[10]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[11]  Luis A. Plana,et al.  SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[12]  Pierre Kornprobst,et al.  Microsaccades enable efficient synchrony-based coding in the retina: a simulation study , 2016, Scientific Reports.

[13]  Garrick Orchard,et al.  Skimming Digits: Neuromorphic Classification of Spike-Encoded Images , 2016, Front. Neurosci..

[14]  Timothée Masquelier,et al.  STDP-based spiking deep neural networks for object recognition , 2016, Neural Networks.

[15]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[16]  Dharmendra S. Modha,et al.  Backpropagation for Energy-Efficient Neuromorphic Computing , 2015, NIPS.

[17]  Steve B. Furber,et al.  Scalable energy-efficient, low-latency implementations of trained spiking Deep Belief Networks on SpiNNaker , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[18]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[19]  Bernabé Linares-Barranco,et al.  An AER handshake-less modular infrastructure PCB with x8 2.5Gbps LVDS serial links , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[20]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[21]  Bernabé Linares-Barranco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Tobi Delbrück,et al.  A Low Power, Fully Event-Based Gesture Recognition System , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Daniel Matolin,et al.  A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS , 2011, IEEE Journal of Solid-State Circuits.

[24]  Garrick Orchard,et al.  Real-time sensory information processing using the TrueNorth Neurosynaptic System , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[25]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..