Fast Pipeline 128×128 pixel spiking convolution core for event-driven vision processing in FPGAs

This paper describes a digital implementation of a parallel and pipelined spiking convolutional neural network (S-ConvNet) core for processing spikes in an event-driven system. Event-driven vision systems use typically as sensor some bio-inspired spiking device, such as the popular Dynamic Vision Sensor (DVS). DVS cameras generate spikes related to changes in light intensity. In this paper we present a 2D convolution event-driven processing core with 128×128 pixels. S-ConvNet is an Event-Driven processing method to extract event features from an input event flow. The nature of spiking systems is highly parallel, in general. Therefore, S-ConvNet processors can benefit from the parallelism offered by Field Programmable Gate Arrays (FPGAs) to accelerate the operation. Using 3 stages of pipeline and a parallel structure, results in updating the state of a 128 neuron row in just 12ns. This improves with respect to previously reported approaches.

[1]  Angel Jiménez-Fernandez,et al.  On the AER convolution processors for FPGA , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[2]  Bernabé Linares-Barranco,et al.  An Event-Driven Multi-Kernel Convolution Processor Module for Event-Driven Vision Sensors , 2012, IEEE Journal of Solid-State Circuits.

[3]  Bernabe Linares-Barranco,et al.  Comparison between Frame-Constrained Fix-Pixel-Value and Frame-Free Spiking-Dynamic-Pixel ConvNets for Visual Processing , 2012, Front. Neurosci..

[4]  Tobi Delbrück,et al.  A 5 Meps $100 USB2.0 Address-Event Monitor-Sequencer Interface , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[5]  Tobias Delbrück,et al.  Frame-free dynamic digital vision , 2008 .

[6]  Misha Anne Mahowald,et al.  VLSI analogs of neuronal visual processing: a synthesis of form and function , 1992 .

[7]  Bernabé Linares-Barranco,et al.  A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Bernabé Linares-Barranco,et al.  An AER handshake-less modular infrastructure PCB with x8 2.5Gbps LVDS serial links , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[9]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[10]  Bernabé Linares-Barranco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Nitish V. Thakor,et al.  HFirst: A Temporal Approach to Object Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Bernabé Linares-Barranco,et al.  A 128$\,\times$ 128 1.5% Contrast Sensitivity 0.9% FPN 3 µs Latency 4 mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers , 2013, IEEE Journal of Solid-State Circuits.

[13]  Andreas G. Andreou,et al.  AER image filtering architecture for vision-processing systems , 1999 .

[14]  Bernabé Linares-Barranco,et al.  A 32$\,\times\,$ 32 Pixel Convolution Processor Chip for Address Event Vision Sensors With 155 ns Event Latency and 20 Meps Throughput , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[15]  Bernabé Linares-Barranco,et al.  Multicasting Mesh AER: A Scalable Assembly Approach for Reconfigurable Neuromorphic Structured AER Systems. Application to ConvNets , 2013, IEEE Transactions on Biomedical Circuits and Systems.

[16]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.