Processing Near Sensor Architecture in Mixed-Signal Domain With CMOS Image Sensor of Convolutional-Kernel-Readout Method

In the era of Artificial Intelligence (AI), bio-inspired perceptual computing system design brings favorable opportunities, while still facing considerable challenges in the meantime. Especially for tasks of image recognition in power-limited vision-based Internet of Things (IoT) devices, energy constraints due to the end of Dennard scaling limit the performance of Neural Network (NN) algorithms on popular digital platforms, which would not reach the energy efficiency requirement for embedded AI applications. In this paper, a processing near sensor architecture in mixed-signal domain with CMOS Image Sensor (CIS) of convolutional-kernel-readout method is proposed. Visual data is collected from a smart CIS, which can realize maximum $5 \times 5$ kernel-readout with minimum one slide step for convolutional operations. The outputs of CIS are directly processed by analog processing units locating near CIS without the constraint of digital clock and bottleneck of Analog-to-Digital Converter (ADC). By analyzing the effects of analog noise on classification accuracy, we further evaluate the fault-tolerance of the system to circuit noise and the device imperfection, such as mismatch and process variation. A mixed-signal visual perception chip is fabricated with a $32\times32$ image sensor and a Binarized Neural Network (BNN) processing array integrated with SMIC 180nm standard CMOS mixed-signal process. Measurement results show up to 545.4 GOPS/W energy efficiency with 1.8mW power consumption taking the advantages of ADC-free processing architecture. This work provides a promising alternative for low-power vision-based IoT intelligent applications.

[1]  Joel Emer,et al.  Eyeriss: an Energy-efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks Accessed Terms of Use , 2022 .

[2]  Jeffrey A. Davis,et al.  The fundamental limit on binary switching energy for terascale integration (TSI) , 2000, IEEE Journal of Solid-State Circuits.

[3]  Liyuan Liu,et al.  A 1000 fps Vision Chip Based on a Dynamically Reconfigurable Hybrid Architecture Comprising a PE Array Processor and Self-Organizing Map Neural Network , 2014, IEEE Journal of Solid-State Circuits.

[4]  Lin Zhong,et al.  RedEye: Analog ConvNet Image Sensor Architecture for Continuous Mobile Vision , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[7]  Minho Kwon,et al.  A 2.1Mpixel 120frame/s CMOS image sensor with column-parallel ΔΣ ADC architecture , 2010, 2010 IEEE International Solid-State Circuits Conference - (ISSCC).

[8]  Jia Wang,et al.  DaDianNao: A Machine-Learning Supercomputer , 2014, 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.

[9]  Angel Rodriguez-Vazquez,et al.  A Bio-Inspired Vision Sensor With Dual Operation and Readout Modes , 2016, IEEE Sensors Journal.

[10]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[11]  Xin He,et al.  NeuADC: Neural Network-Inspired Synthesizable Analog-to-Digital Conversion , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[12]  Hongbo Zhu,et al.  A real-time motion-feature-extraction image processor employing digital-pixel-sensor-based parallel architecture , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[13]  Qi Wei,et al.  CMOS Image Sensor Data-Readout Method for Convolutional Operations with Processing Near Sensor Architecture , 2018, 2018 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).

[14]  Minho Kwon,et al.  A 2.1 M Pixels, 120 Frame/s CMOS Image Sensor With Column-Parallel $\Delta \Sigma$ ADC Architecture , 2011, IEEE Journal of Solid-State Circuits.

[15]  R.H. Dennard,et al.  Design Of Ion-implanted MOSFET's with Very Small Physical Dimensions , 1974, Proceedings of the IEEE.

[16]  Xuan Zhang,et al.  NeuADC: Neural Network-Inspired RRAM-Based Synthesizable Analog-to-Digital Conversion with Reconfigurable Quantization Support , 2019, 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[17]  Shahram Shirani,et al.  CMOS Image Sensor With Area-Efficient Block-Based Compressive Sensing , 2015, IEEE Sensors Journal.

[18]  Engin Ipek,et al.  Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[19]  Tianshi Chen,et al.  ShiDianNao: Shifting vision processing closer to the sensor , 2015, 2015 ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA).

[20]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  John P. Hayes,et al.  Energy-efficient hybrid stochastic-binary neural networks for near-sensor computing , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[22]  Arash Ardakani,et al.  A Convolutional Accelerator for Neural Networks With Binary Weights , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[23]  Steven R. Young,et al.  A 1 TOPS/W Analog Deep Machine-Learning Engine With Floating-Gate Storage in 0.13 µm CMOS , 2014, IEEE Journal of Solid-State Circuits.

[24]  Iluminada Baturone,et al.  Current-mode multiple-input Max circuit , 1994 .

[25]  Ninghui Sun,et al.  DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning , 2014, ASPLOS.

[26]  Tadashi Shibata,et al.  A Computational Digital Pixel Sensor Featuring Block-Readout Architecture for On-Chip Image Processing , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[27]  Warren J. Gross,et al.  An Architecture to Accelerate Convolution in Deep Neural Networks , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[28]  Shoushun Chen,et al.  A Two-Step Prediction ADC Architecture for Integrated Low Power Image Sensors , 2017, IEEE Transactions on Circuits and Systems I: Regular Papers.