Utilizing Direct Photocurrent Computation and 2D Kernel Scheduling to Improve In-Sensor-Processing Efficiency

Deploying intelligent visual algorithms in terminal devices for always-on sensing is an attractive trend in the IoT era. In-sensor-processing architecture is proposed to reduce power consumption on A/D conversion and data transmission, which performs pre-processing and only converting low-throughput features. However, current designs still require high energy consumption on photoelectric conversion and analog data movement. In this paper, two methods are proposed to improve the energy efficiency of in-sensor-processing architecture, including direct photocurrent computation and 2D kernel scheduling. Photocurrents are directly involved in computation to avoid data conversion; thus the indispensable imaging power is also utilized for computing. Since the location of the pixel data is fixed, data scheduling is conducted on digital weights to eliminate analog data storage and movement. We implement a prototype chip with an array of 32 × 32 units to calculate the first layer of binarized LeNet-5. The post-simulation shows that the proposed architecture reaches the energy efficiency of 11.49TOPs/W, about 14.8x higher than previous works.

[1]  Shida Sayaka,et al.  A 1ms High-Speed Vision Chip with 3D-Stacked 140GOPS Column-Parallel PEs for Spatio-Temporal Image Processing , 2017 .

[2]  Seokjun Park,et al.  A 272.49 pJ/pixel CMOS image sensor with embedded object detection and bio-inspired 2D optic flow generation for nano-air-vehicle navigation , 2017, 2017 Symposium on VLSI Circuits.

[3]  Jaehyuk Choi,et al.  Always-On CMOS Image Sensor for Mobile and Wearable Devices , 2016, IEEE Journal of Solid-State Circuits.

[4]  Meng-Fan Chang,et al.  A 0.5V Real-Time Computational CMOS Image Sensor with Programmable Kernel for Always-On Feature Extraction , 2019, 2019 IEEE Asian Solid-State Circuits Conference (A-SSCC).

[5]  Desoli Mr Giuseppe,et al.  14.1 A 2.9TOPS/W deep convolutional neural network SoC in FD-SOI 28nm for intelligent embedded systems , 2017 .

[6]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[7]  Ran El-Yaniv,et al.  Binarized Neural Networks , 2016, ArXiv.

[8]  Seokjun Park,et al.  7.2 243.3pJ/pixel bio-inspired time-stamp-based 2D optic flow sensor for artificial compound eyes , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[9]  Xin Wang,et al.  5.1 A Stacked Global-Shutter CMOS Imager with SC-Type Hybrid-GS Pixel and Self-Knee Point Calibration Single Frame HDR and On-Chip Binarization Algorithm for Smart Vision Applications , 2019, 2019 IEEE International Solid- State Circuits Conference - (ISSCC).

[10]  Masatoshi Ishikawa,et al.  4.9 A 1ms high-speed vision chip with 3D-stacked 140GOPS column-parallel PEs for spatio-temporal image processing , 2017, 2017 IEEE International Solid-State Circuits Conference (ISSCC).

[11]  Hoi-Jun Yoo,et al.  A Low-Power Convolutional Neural Network Face Recognition Processor and a CIS Integrated With Always-on Face Detector , 2018, IEEE Journal of Solid-State Circuits.

[12]  Wancheng Zhang,et al.  A Programmable SIMD Vision Chip for Real-Time Vision Applications , 2008, IEEE Journal of Solid-State Circuits.

[13]  Marian Verhelst,et al.  An always-on 3.8μJ/86% CIFAR-10 mixed-signal binary CNN processor with all memory on chip in 28nm CMOS , 2018, 2018 IEEE International Solid - State Circuits Conference - (ISSCC).