A hierarchical vision processing architecture oriented to 3D integration of smart camera chips

This paper introduces a vision processing architecture that is directly mappable on a 3D chip integration technology. Due to the aggregated nature of the information contained in the visual stimulus, adapted architectures are more efficient than conventional processing schemes. Given the relatively minor importance of the value of an isolated pixel, converting every one of them to digital prior to any processing is inefficient. Instead of this, our system relies on focal-plane image filtering and key point detection for feature extraction. The originally large amount of data representing the image is now reduced to a smaller number of abstracted entities, simplifying the operation of the subsequent digital processor. There are certain limitations to the implementation of such hierarchical scheme. The incorporation of processing elements close to the photo-sensing devices in a planar technology has a negative influence in the fill factor, pixel pitch and image size. It therefore affects the sensitivity and spatial resolution of the image sensor. A fundamental tradeoff needs to be solved. The larger the amount of processing conveyed to the sensor plane, the larger the pixel pitch. On the contrary, using a smaller pixel pitch sends more processing circuitry to the periphery of the sensor and tightens the data bottleneck between the sensor plane and the memory plane. 3D integration technologies with a high density of through-silicon-vias can help overcome these limitations. Vertical integration of the sensor plane and the processing and memory planes with a fully parallel connection eliminates data bottlenecks without compromising fill factor and pixel pitch. A case study is presented: a smart vision chip designed on a 3D integration technology provided by MIT Lincoln Labs, whose base process is 0.15@mm FD-SOI. Simulation results advance performance improvements with respect to the state-of-the-art in smart vision chips.

[1]  Wancheng Zhang,et al.  A Programmable Vision Chip Based on Multiple Levels of Parallel Processors , 2011, IEEE Journal of Solid-State Circuits.

[2]  Hoi-Jun Yoo,et al.  24-GOPS 4.5-${\rm mm}^{2}$ Digital Cellular Neural Network for Rapid Visual Attention in an Object-Recognition SoC , 2011, IEEE Transactions on Neural Networks.

[3]  Boyd Fowler,et al.  Low-FPN high-gain capacitive transimpedance amplifier for low-noise CMOS image sensors , 2001, IS&T/SPIE Electronic Imaging.

[4]  Botond Roska,et al.  Parallel processing in retinal ganglion cells: how integration of space-time patterns of excitation and inhibition form the spiking output. , 2006, Journal of neurophysiology.

[5]  Ángel Rodríguez-Vázquez,et al.  Toward visual microprocessors , 2002, Proc. IEEE.

[6]  Belliappa Kuttanna,et al.  A Sub-1W to 2W Low-Power IA Processor for Mobile Internet Devices and Ultra-Mobile PCs in 45nm Hi-Κ Metal Gate CMOS , 2008, 2008 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[7]  D. B. Davis,et al.  Intel Corp. , 1993 .

[8]  Ricardo Carmona-Galán,et al.  FLIP-Q: A QCIF Resolution Focal-Plane Array for Low-Power Image Processing , 2011, IEEE Journal of Solid-State Circuits.

[9]  Ángel Rodríguez-Vázquez,et al.  In-pixel ADC for a vision architecture on CMOS-3D technology , 2010, 2010 IEEE International 3D Systems Integration Conference (3DIC).

[10]  Piotr Dudek,et al.  A general-purpose processor-per-pixel analog SIMD vision chip , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[11]  Peter Ramm,et al.  Handbook of 3D integration : technology and applications of 3D integrated circuits , 2012 .

[12]  José González,et al.  Distributed Cooperative Caching: An Energy Efficient Memory Scheme for Chip Multiprocessors , 2012, IEEE Transactions on Parallel and Distributed Systems.

[13]  John L. Gustafson,et al.  Reevaluating Amdahl's law , 1988, CACM.

[14]  Brian A. Wandell,et al.  How small should pixel size be? , 2000, Electronic Imaging.

[15]  Yun-Tae Lee,et al.  A 1/2.33-inch 14.6M 1.4μm-pixel backside-illuminated CMOS image sensor with floating diffusion boosting , 2011, 2011 IEEE International Solid-State Circuits Conference.

[16]  Angel Rodriguez-Vazquez,et al.  Offset-compensated comparator with full-input range in 150nm FDSOI CMOS-3D technology , 2010, 2010 First IEEE Latin American Symposium on Circuits and Systems (LASCAS).

[17]  Thomas D. Burd,et al.  Processor design for portable systems , 1996, J. VLSI Signal Process..

[18]  Francisco Sandoval,et al.  Multifoveal imager for stereo applications , 2002, Int. J. Imaging Syst. Technol..

[19]  Viktor K. Prasanna,et al.  High-performance computing for vision , 1996, Proc. IEEE.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Gabor C. Temes,et al.  Circuit techniques for reducing the effects of op-amp imperfections: autozeroing, correlated double sampling, and chopper stabilization , 1996, Proc. IEEE.

[22]  Ángel Rodríguez-Vázquez,et al.  A QCIF 145dB imager for focal plane processor chips using a tone mapping technique in standard 0.35μm CMOS technology , 2011 .

[23]  Gustavo Liñán Cembrano,et al.  A 1000 FPS at 128×128 vision processor with 8-bit digitized I/O , 2004, IEEE J. Solid State Circuits.

[24]  Ákos Zarándy,et al.  2D operators on topographic and non‐topographic architectures—implementation, efficiency analysis, and architecture selection methodology , 2011, Int. J. Circuit Theory Appl..

[25]  S. Burkett,et al.  Process integration for through-silicon vias , 2005 .

[26]  M. Gottardi,et al.  A 100 $\mu$ W 128 $\times$ 64 Pixels Contrast-Based Asynchronous Binary Vision Sensor for Sensor Networks Applications , 2009, IEEE Journal of Solid-State Circuits.

[27]  Paul Wielage,et al.  XETAL-II: A 107 GOPS, 600mW Massively-Parallel Processor for Video Scene Analysis , 2007, 2007 IEEE International Solid-State Circuits Conference. Digest of Technical Papers.

[28]  Wancheng Zhang,et al.  A Programmable SIMD Vision Chip for Real-Time Vision Applications , 2008, IEEE Journal of Solid-State Circuits.

[29]  Danny Crookes,et al.  Parallel architectures for image processing , 1998 .

[30]  Tamás Roska,et al.  Digital implementation of cellular sensor-computers: Research Articles , 2006 .

[31]  István Petrás,et al.  A bio-inspired two-layer mixed-signal flexible programmable chip for early vision , 2003, IEEE Trans. Neural Networks.

[32]  Csaba Rekeczky,et al.  3D multi-layer vision architecture for surveillance and reconnaissance applications , 2009, 2009 European Conference on Circuit Theory and Design.

[33]  Xinqiao Liu,et al.  A 10000 frames/s CMOS digital pixel sensor , 2001, IEEE J. Solid State Circuits.

[34]  A. Zarandy,et al.  Displacement calculation algorithm on a heterogenious multi-layer cellular sensor processor array , 2010, 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010).

[35]  Luca Benini,et al.  Synthesis of application-specific memories for power optimization in embedded systems , 2000, Proceedings 37th Design Automation Conference.

[36]  Piotr Dudek,et al.  A SIMD Cellular Processor Array Vision Chip With Asynchronous Processing Capabilities , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[37]  Dan Schonfeld,et al.  A new image representation algorithm inspired by image submodality models, redundancy reduction, and learning in biological vision , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  R. Masland.  The fundamental plan of the retina , 2001, Nature Neuroscience.

[39]  R.P. Kleihorst,et al.  Xetal-II: A 107 GOPS, 600 mW Massively Parallel Processor for Video Scene Analysis , 2008, IEEE Journal of Solid-State Circuits.

[40]  S. Espejo,et al.  A 1000 FPS at 128/spl times/128 vision processor with 8-bit digitized I/O , 2004, IEEE Journal of Solid-State Circuits.

[41]  Tamás Roska,et al.  Digital implementation of cellular sensor-computers , 2006, Int. J. Circuit Theory Appl..

[42]  Mark D. Hill,et al.  Amdahl's Law in the Multicore Era , 2008 .

[43]  J. Romberg,et al.  Imaging via Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[44]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  M. Ishikawa,et al.  A dynamically reconfigurable SIMD processor for a vision chip , 2003, IEEE Journal of Solid-State Circuits.

[46]  Wayne Luk,et al.  Binomial filters , 1996, J. VLSI Signal Process..

[47]  F. Werblin,et al.  Vertical interactions across ten parallel, stacked representations in the mammalian retina , 2001, Nature.

[48]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[49]  C.K. Chen,et al.  A wafer-scale 3-D circuit integration technology , 2006, IEEE Transactions on Electron Devices.

[50]  Benjamin W. Wah,et al.  Wiley Encyclopedia of Computer Science and Engineering , 2009, Wiley Encyclopedia of Computer Science and Engineering.

[51]  Ángel Rodríguez-Vázquez,et al.  Switched-capacitor networks for scale-space generation , 2011, 2011 20th European Conference on Circuit Theory and Design (ECCTD).

[52]  Ángel Rodríguez-Vázquez,et al.  ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[53]  Ulrich Ramacher,et al.  A 100-GOPS programmable processor for vehicle vision systems , 2003, IEEE Design & Test of Computers.

[54]  A. Rodriguez-Vazquez,et al.  Digital processor array implementation aspects of a 3D multi-layer vision architecture , 2010, 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010).