A 1000 fps Vision Chip Based on a Dynamically Reconfigurable Hybrid Architecture Comprising a PE Array Processor and Self-Organizing Map Neural Network

This paper proposes a vision chip hybrid architecture with dynamically reconfigurable processing element (PE) array processor and self-organizing map (SOM) neural network. It integrates a high speed CMOS image sensor, three von Neumann-type processors, and a non-von Neumann-type bio-inspired SOM neural network. The processors consist of a pixel-parallel PE array processor with O(N×N) parallelism, a row-parallel row-processor (RP) array processor with O(N) parallelism and a thread-parallel dual-core microprocessor unit (MPU) with O(2) parallelism. They execute low-, mid- and high-level image processing, respectively. The SOM network speeds up high-level processing in pattern recognition tasks by O(N/4×N/4), which improves the chip performance remarkably. The SOM network can be dynamically reconfigured from the PE array to largely save chip area. A prototype chip with a 256 × 256 image sensor, a reconfigurable 64 × 64 PE array processor/16 × 16 SOM network, a 64 × 1 RP array processor and a dual-core 32-bit MPU was implemented in a 0.18 μm CMOS image sensor process. The chip can perform image capture and various-level image processing at a high speed and in flexible fashion. Various complicated applications including M-S functional solution, horizon estimation, hand gesture recognition, face recognition are demonstrated at high speed from several hundreds to >1000 fps.

[1]  Hoi-Jun Yoo,et al.  A 320mW 342GOPS real-time moving object recognition processor for HD 720p video streams , 2012, 2012 IEEE International Solid-State Circuits Conference.

[2]  M. Ikeda,et al.  A 375 /spl times/ 365 high-speed 3-D range-finding image sensor using row-parallel search architecture and multisampling technique , 2005, IEEE Journal of Solid-State Circuits.

[3]  Piotr Dudek,et al.  A SIMD Cellular Processor Array Vision Chip With Asynchronous Processing Capabilities , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[4]  Stanislaw Szczepanski,et al.  An Analog Sub-Miliwatt CMOS Image Sensor With Pixel-Level Convolution Processing , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  Shoji Kawahito,et al.  A 33Mpixel 120fps CMOS image sensor using 12b column-parallel pipelined cyclic ADCs , 2012, 2012 IEEE International Solid-State Circuits Conference.

[6]  Wancheng Zhang,et al.  A Programmable SIMD Vision Chip for Real-Time Vision Applications , 2008, IEEE Journal of Solid-State Circuits.

[7]  Ángel Rodríguez-Vázquez,et al.  ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  David Stoppa,et al.  A CMOS image sensor with programmable pixel-level analog processing , 2005, IEEE Transactions on Neural Networks.

[9]  M. Ishikawa,et al.  A dynamically reconfigurable SIMD processor for a vision chip , 2003, IEEE Journal of Solid-State Circuits.

[10]  Donghyun Kim,et al.  A 125GOPS 583mW Network-on-Chip Based Parallel Processor with Bio-inspired Visual-Attention Engine , 2008, 2008 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[11]  Chih-Chi Cheng,et al.  iVisual: An Intelligent Visual Sensor SoC With 2790 fps CMOS Image Sensor and 205 GOPS/W Vision Processor , 2009, IEEE Journal of Solid-State Circuits.

[12]  T. Shibata,et al.  A Real-Time Image-Feature-Extraction and Vector-Generation VLSI Employing Arrayed-Shift-Register Architecture , 2007, IEEE Journal of Solid-State Circuits.

[13]  Piotr Dudek,et al.  A general-purpose processor-per-pixel analog SIMD vision chip , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[14]  M. Gottardi,et al.  A 100 dB Dynamic-Range CMOS Vision Sensor With Programmable Image Processing and Global Feature Extraction , 2007, IEEE Journal of Solid-State Circuits.

[15]  Shoji Kawahito,et al.  An 80μVrms-temporal-noise 82dB-dynamic-range CMOS Image Sensor with a 13-to-19b variable-resolution column-parallel folding-integration/cyclic ADC , 2011, 2011 IEEE International Solid-State Circuits Conference.

[16]  Timothy K. Horiuchi.  A Low-Power Visual-Horizon Estimation Chip , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[17]  Bernabé Linares-Barranco,et al.  A 3.6 $\mu$ s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor , 2011, IEEE Journal of Solid-State Circuits.

[18]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[19]  Bo Zhao,et al.  A 64$\,\times\,$64 CMOS Image Sensor With On-Chip Moving Object Detection and Localization , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Liang-Gee Chen,et al.  iVisual: An intelligent visual sensor SoC with 2790fps CMOS image sensor and 205GOPS/W vision processor , 2008, 2008 45th ACM/IEEE Design Automation Conference.

[21]  D. C. Hendry,et al.  IP core implementation of a self-organizing neural network , 2003, IEEE Trans. Neural Networks.

[22]  Michael W. Hoffman,et al.  A CMOS Imager With Focal Plane Compression Using Predictive Coding , 2007, IEEE Journal of Solid-State Circuits.

[23]  Bernabé Linares-Barranco,et al.  A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[24]  Gunhee Han,et al.  A 200 $\mu$ s Processing Time Smart Image Sensor for an Eye Tracker Using Pixel-Level Analog Image Processing , 2009, IEEE Journal of Solid-State Circuits.

[25]  Nanjian Wu,et al.  A 10-bit column-parallel cyclic ADC for high-speed CMOS image sensors , 2013 .

[26]  Wancheng Zhang,et al.  A 1,000 Frames/s Programmable Vision Chip with Variable Resolution and Row-Pixel-Mixed Parallel Image Processors , 2009, Sensors.

[27]  Fan Yang,et al.  Implementation of an RBF neural network on embedded systems: real-time face tracking and identity verification , 2003, IEEE Trans. Neural Networks.

[28]  M. Ishikawa,et al.  A CMOS vision chip with SIMD processing element array for 1 ms image processing , 1999, 1999 IEEE International Solid-State Circuits Conference. Digest of Technical Papers. ISSCC. First Edition (Cat. No.99CH36278).

[29]  S. Himavathi,et al.  Feedforward Neural Network Implementation in FPGA Using Layer Multiplexing for Effective Resource Utilization , 2007, IEEE Transactions on Neural Networks.

[30]  Kiyoharu Aizawa,et al.  Computational Sensors - Vision VLSI , 1999 .

[31]  Nicolas D. Georganas,et al.  Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques , 2011, IEEE Transactions on Instrumentation and Measurement.

[32]  Wancheng Zhang,et al.  A Programmable Vision Chip Based on Multiple Levels of Parallel Processors , 2011, IEEE Journal of Solid-State Circuits.

[33]  Joo-Young Kim,et al.  A 125 GOPS 583 mW Network-on-Chip Based Parallel Processor With Bio-Inspired Visual Attention Engine , 2009, IEEE Journal of Solid-State Circuits.

[34]  M. Paindavoine,et al.  A 10 000 fps CMOS Sensor With Massively Parallel Image Processing , 2008, IEEE Journal of Solid-State Circuits.

[35]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[36]  Jau-Hsiung Huang,et al.  On Parallel Processing Systems: Amdahl's Law Generalized and Some Results on Optimal Design , 1992, IEEE Trans. Software Eng..

[37]  Yehoshua Y. Zeevi,et al.  Image enhancement and denoising by complex diffusion processes , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Idaku Ishii,et al.  A digital vision chip specialized for high-speed target tracking , 2003 .