Neuromorphic Image Sensor Design with Region-Aware Processing

This paper presents a pixel parallel architecture of a neuromorphic image sensor, designed as a 3D bottom-up architecture composing of several computational planes where each plane performs different image processing algorithms. The model emulates the hierarchical process in biological vision by providing feedforward and feedback information flow between different planes. The on-chip attention module dynamically detects regions with relevant information and produces a feedback path to sample those regions with a higher clock frequency, whereas regions with low spatial and temporal information receive less attention. The results suggest that by sampling non-relevant regions with a lower frequency, the sensor can reduce redundancy and enable high-performance computing at low power. Furthermore, by deploying high-level reasoning only on the selected regions instead of the entire image the model can decrease computational expenses.

[1]  Senem Velipasalar,et al.  Cooperative Object Tracking and Composite Event Detection With Wireless Embedded Smart Cameras , 2010, IEEE Transactions on Image Processing.

[2]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[3]  Christophe Bobda,et al.  Pixel-Parallel Architecture for Neuromorphic Smart Image Sensor with Visual Attention , 2018, 2018 IEEE Computer Society Annual Symposium on VLSI (ISVLSI).

[4]  Marjan Asadinia,et al.  Design of a Reconfigurable 3D Pixel-Parallel Neuromorphic Architecture for Smart Image Sensor , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Christophe Bobda,et al.  SoC Design Of A Novel Cluster-Based Approach for Real-Time Lane Detection in Low Quality Images , 2017, ICDSC.

[7]  Martin Vetterli,et al.  Event-driven video coding for outdoor wireless monitoring cameras , 2012, 2012 19th IEEE International Conference on Image Processing.

[8]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[9]  B. Tyrrell,et al.  Time Delay Integration and In-Pixel Spatiotemporal Filtering Using a Nanoscale Digital CMOS Focal Plane Readout , 2009, IEEE Transactions on Electron Devices.

[10]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[11]  David Blaauw,et al.  Low-Power and Compact Analog-to-Digital Converter Using Spintronic Racetrack Memory Devices , 2017, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[12]  A. Dupret,et al.  A Smart Sensor for Image Processing: Towards a System on Chip , 2006, 2006 IEEE International Symposium on Industrial Electronics.

[13]  Giacomo Indiveri,et al.  A neuromorphic saliency-map based active vision system , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[14]  Peter Elias,et al.  Predictive coding-I , 1955, IRE Trans. Inf. Theory.

[15]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Ákos Zarándy,et al.  Focal-Plane Sensor-Processor Chips , 2014 .

[17]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[18]  Rajesh P. N. Rao,et al.  Predictive Coding , 2019, A Blueprint for the Hard Problem of Consciousness.

[19]  Michael DeBole,et al.  Accelerators for biologically-inspired attention and recognition , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).