Hierarchical and Parallel Pipelined Heterogeneous SoC for Embedded Vision Processing

Object recognition is widely used in vision computing for various applications. Traditional CPU and application specific integrated circuit for vision computing cannot provide high performance and enough flexibility, which limit the use of vision systems. In this paper, a hierarchical and parallel pipelined heterogeneous chip for object recognition is proposed to achieve high flexibility, high performance, and area efficiency. In addition, a reformulation of 3D position estimation is proposed. The method uses single precision to achieve the short computing time and accuracy requirement. The hardware resource is small. Application-specific components, such as connected component information extractor and information extraction accelerator, are designed for high performance. Reconfiguration processors and application-specific instruction set processor are introduced to improve flexibility. These components are connected to hierarchical parallel buses. The chip is fabricated in 180-nm CMOS technology and occupies 72.25 mm2 with 1.09M bits on-chip memory. It delivers 204 GOPS + 665M FLOPS operations. The results show that this hierarchical and parallel pipelined heterogeneous chip is suitable for embedded vision systems.

[1]  Byeongchan Jeon,et al.  A Memory-Efficient Architecture of Full HD Around View Monitor Systems , 2014, IEEE Transactions on Intelligent Transportation Systems.

[2]  Liang-Gee Chen,et al.  A 52 mW Full HD 160-Degree Object Viewpoint Recognition SoC With Visual Vocabulary Processor for Wearable Vision Applications , 2012, IEEE Journal of Solid-State Circuits.

[3]  Hoi-Jun Yoo,et al.  A 320 mW 342 GOPS Real-Time Dynamic Object Recognition Processor for HD 720p Video Streams , 2013, IEEE Journal of Solid-State Circuits.

[4]  查正军,et al.  A Unified Scheme for Super-resolution and Depth Estimation from Asymmetric Stereoscopic Video , 2016 .

[5]  Javier Díaz,et al.  Real-Time Model-Based Rigid Object Pose Estimation and Tracking Combining Dense and Sparse Visual Cues , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Hoi-Jun Yoo,et al.  A 57mW embedded mixed-mode neuro-fuzzy accelerator for intelligent multi-core processor , 2011, 2011 IEEE International Solid-State Circuits Conference.

[8]  Hongbo Zhu,et al.  A Real-Time Motion-Feature-Extraction VLSI Employing Digital-Pixel-Sensor-Based Parallel Architecture , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Piotr Dudek,et al.  A general-purpose processor-per-pixel analog SIMD vision chip , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[10]  Liyuan Liu,et al.  A 1000 fps Vision Chip Based on a Dynamically Reconfigurable Hybrid Architecture Comprising a PE Array Processor and Self-Organizing Map Neural Network , 2014, IEEE Journal of Solid-State Circuits.

[11]  Idaku Ishii,et al.  A digital vision chip specialized for high-speed target tracking , 2003 .

[12]  Sukhan Lee,et al.  Embedded Visual SLAM: Applications for Low-Cost Consumer Robots , 2013, IEEE Robotics & Automation Magazine.

[13]  Arthur Daniel Costea,et al.  Word Channel Based Multiscale Pedestrian Detection without Image Resizing and Using Only One Classifier , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Wancheng Zhang,et al.  A Programmable Vision Chip Based on Multiple Levels of Parallel Processors , 2011, IEEE Journal of Solid-State Circuits.

[15]  Luca Benini,et al.  Platform 2012, a many-core computing accelerator for embedded SoCs: Performance evaluation of visual analytics applications , 2012, DAC Design Automation Conference 2012.

[16]  Siddhartha S. Srinivasa,et al.  MOPED: A scalable and low latency object recognition and pose estimation system , 2010, 2010 IEEE International Conference on Robotics and Automation.

[17]  Piotr Dudek,et al.  A SIMD Cellular Processor Array Vision Chip With Asynchronous Processing Capabilities , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[18]  Luc Van Gool,et al.  Pedestrian detection at 100 frames per second , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  John L. Wyatt,et al.  The MIT Vision Chip Project: analog VLSI systems for fast image acquisition and early vision processing , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[20]  Jizhong Zhao,et al.  Matrix computing coprocessor for an embedded system , 2015, Microprocess. Microsystems.

[21]  Andrew W. Fitzgibbon.  Robust registration of 2D and 3D point sets , 2003, Image Vis. Comput..

[22]  Richard P. Wildes,et al.  Action Spotting and Recognition Based on a Spatiotemporal Orientation Analysis , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  C. Toumazou,et al.  A micropower centroiding vision processor , 2006, IEEE Journal of Solid-State Circuits.

[24]  Mo-Yuen Chow,et al.  Field-Programmable System-on-Chip for Localization of UGVs in an Indoor iSpace , 2014, IEEE Transactions on Industrial Informatics.

[25]  Hyo-Eun Kim,et al.  A 182 mW 94.3 f/s in Full HD Pattern-Matching Based Image Recognition Accelerator for an Embedded Vision System in 0.13-$\mu{\rm m}$ CMOS Technology , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Y. M. Mustafah,et al.  Skin region detector for real time face detection system , 2012, 2012 International Conference on Computer and Communication Engineering (ICCCE).

[27]  Eduardo Ros,et al.  Massive Parallel-Hardware Architecture for Multiscale Stereo, Optical Flow and Image-Structure Computation , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[29]  Janne Heikkilä,et al.  Geometric Camera Calibration Using Circular Control Points , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Shao-Yi Chien,et al.  VLSI Architecture Design of Guided Filter for 30 Frames/s Full-HD Video , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Bin Wang,et al.  A 100,000 fps vision sensor with embedded 535GOPS/W 256×256 SIMD processor array , 2013, 2013 Symposium on VLSI Circuits.

[32]  Chih-Chi Cheng,et al.  iVisual: An Intelligent Visual Sensor SoC With 2790 fps CMOS Image Sensor and 205 GOPS/W Vision Processor , 2009, IEEE Journal of Solid-State Circuits.

[33]  William J. Dally,et al.  Imagine: Media Processing with Streams , 2001, IEEE Micro.

[34]  Hoi-Jun Yoo,et al.  A 201.4 GOPS 496 mW Real-Time Multi-Object Recognition Processor With Bio-Inspired Neural Perception Engine , 2009, IEEE Journal of Solid-State Circuits.

[35]  Donghyun Kim,et al.  A 201.4GOPS 496mW real-time multi-object recognition processor with bio-inspired neural perception engine , 2009, 2009 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[36]  Hoi-Jun Yoo,et al.  A 345 mW Heterogeneous Many-Core Processor With an Intelligent Inference Engine for Robust Object Recognition , 2011, IEEE J. Solid State Circuits.

[37]  Derek Abbott,et al.  An insect vision-based motion detection chip , 1997, IEEE J. Solid State Circuits.

[38]  Liang-Gee Chen,et al.  A 1.0TOPS/W 36-core neocortical computing processor with 2.3Tb/s Kautz NoC for universal visual recognition , 2012, 2012 IEEE International Solid-State Circuits Conference.

[39]  Babette Dellen,et al.  Real-Time Segmentation of Stereo Videos on a Portable System With a Mobile GPU , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Yutaka Yamada,et al.  18.2 A 1.9TOPS and 564GOPS/W heterogeneous multicore SoC with color-based object classification accelerator for image-recognition applications , 2015, 2015 IEEE International Solid-State Circuits Conference - (ISSCC) Digest of Technical Papers.

[41]  Nanning Zheng,et al.  Coarse-grained Dynamically Reconfigurable Processor for Vision Pre-Processing , 2015, J. Signal Process. Syst..

[42]  Takashi Miyamori,et al.  A 464GOPS 620GOPS/W heterogeneous multi-core SoC for image-recognition applications , 2012, 2012 IEEE International Solid-State Circuits Conference.

[43]  Liang-Gee Chen,et al.  A 69mW 140-meter/60fps and 60-meter/300fps intelligent vision SoC for versatile automotive applications , 2012, 2012 Symposium on VLSI Circuits (VLSIC).

[44]  Brian C. Lovell,et al.  Real-Time Face Detection and Tracking for High Resolution Smart Camera System , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).

[45]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[46]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).