An SIMD Programmable Vision Chip with High-Speed Focal Plane Image Processing

A high-speed analog VLSI image acquisition and low-level image processing system are presented. The architecture of the chip is based on a dynamically reconfigurable SIMD processor array. The chip features a massively parallel architecture enabling the computation of programmable mask-based image processing in each pixel. Extraction of spatial gradients and convolutions such as Sobel operators are implemented on the circuit. Each pixel includes a photodiode, an amplifier, two storage capacitors, and an analog arithmetic unit based on a four-quadrant multiplier architecture. A pixel proof-of-concept chip was fabricated in a 0.35 m standard CMOS process, with a pixel size of 35 m 35 m. A dedicated embedded platform including FPGA and ADCs has also been designed to evaluate the vision chip. The chip can capture raw images up to 10 000 frames per second and runs low-level image processing at a framerate of 2 000 to 5 000 frames per second.

[1]  Orly Yadid-Pecht,et al.  CMOS active pixel sensor star tracker with regional electronic shutter , 1997 .

[2]  Thierry M. Bernard,et al.  Second generation programmable artificial retina , 1999, Twelfth Annual IEEE International ASIC/SOC Conference (Cat. No.99TH8454).

[3]  Christophe Garcia,et al.  Convolutional face finder: a neural architecture for fast and robust face detection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Bruce A. Wooley,et al.  A 128×128-pixel standard-CMOS image sensor with electronic shutter , 1996 .

[5]  Piotr Dudek.  Implementation of SIMD vision chip with 128/spl times/128 array of analogue processing elements , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[6]  Hae-Seung Lee,et al.  A mixed-signal array processor with early vision applications , 1997, Proceedings of CICC 97 - Custom Integrated Circuits Conference.

[7]  L. Blanquart,et al.  Pixel noise suppression via SoC management of tapered reset in a 1920/spl times/1080 CMOS image sensor , 2005, IEEE Journal of Solid-State Circuits.

[8]  Abbas El Gamal,et al.  Photocurrent estimation from multiple nondestructive samples in CMOS image sensor , 2001, IS&T/SPIE Electronic Imaging.

[9]  S. Espejo,et al.  A 1000 FPS at 128/spl times/128 vision processor with 8-bit digitized I/O , 2004, IEEE Journal of Solid-State Circuits.

[10]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[11]  Shen-Iuan Liu,et al.  CMOS squarer and four-quadrant multiplier , 1995 .

[12]  C. R. Ryan.  Applications of a four-quadrant multiplier , 1970 .

[13]  Ángel Rodríguez-Vázquez,et al.  ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[14]  Ralph Etienne-Cummings,et al.  A programmable focal-plane MIMD image processor chip , 2001, IEEE J. Solid State Circuits.

[15]  Abbas El Gamal,et al.  Pixel-level processing: why, what, and how? , 1999, Electronic Imaging.

[16]  Xinqiao Liu,et al.  Simultaneous image formation and motion blur restoration via multiple capture , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[17]  Piotr Dudek,et al.  An analogue SIMD focal-plane processor array , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[18]  Chung-Yu Wu,et al.  Design, optimization, and performance analysis of new photodiode structures for CMOS active-pixel-sensor (APS) imager applications , 2004, IEEE Sensors Journal.

[19]  K. Azadet,et al.  A 200 mW 3.3 V CMOS color camera IC producing 352/spl times/288 24 b video at 30 frames/s , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[20]  Eric R. Fossum,et al.  128 x 128 CMOS photodiode-type active pixel sensor with on-chip timing, control, and signal chain electronics , 1995, Electronic Imaging.

[21]  A. Herms,et al.  A 128/spl times/128 CMOS image sensor with analog memory for synchronous image capture , 2002 .

[22]  Barry Fowler,et al.  A 640×512 CMOS Image Sensor with Ultra Wide Dynamic Range Floating-Point Pixel-Level ADC , 1999 .

[23]  Piotr Dudek,et al.  General-purpose 128×128 SIMD processor array with integrated image sensor , 2006 .

[24]  Orly Yadid-Pecht,et al.  In-pixel autoexposure CMOS APS , 2003 .

[25]  M. Panaghiston,et al.  A single-chip 306/spl times/244-pixel CMOS NTSC video camera , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[26]  Orly Yadid-Pecht,et al.  Photoresponse analysis and pixel shape optimization for CMOS active pixel sensors , 2003 .

[27]  Fan Yang,et al.  A Parallel Face Detection System Implemented on FPGA , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[28]  Fan Yang,et al.  Implementation of an RBF neural network on embedded systems: real-time face tracking and identity verification , 2003, IEEE Trans. Neural Networks.

[29]  Eric R. Fossum,et al.  Active pixel sensors: are CCDs dinosaurs? , 1993, Electronic Imaging.

[30]  B.J. Hosticka,et al.  A CMOS image sensor for high-speed imaging , 2000, 2000 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.00CH37056).

[31]  Kwangho Yoon,et al.  Single-chip CMOS image sensor for mobile applications , 2002, 2002 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.02CH37315).

[32]  Dave Litwiller,et al.  CCD vs. CMOS: Facts and Fiction , 2001 .

[33]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera on a chip , 1995, Proceedings of International Electron Devices Meeting.

[34]  A. Matsuzawa,et al.  A CMOS image sensor for focal-plane low-power motion vector estimation , 2000, 2000 Symposium on VLSI Circuits. Digest of Technical Papers (Cat. No.00CH37103).

[35]  M. Paindavoine,et al.  A 10 000 fps CMOS Sensor With Massively Parallel Image Processing , 2008, IEEE Journal of Solid-State Circuits.

[36]  Alexander I. Krymski,et al.  A 9V / Lux-s 5000-Frames / s 512 512 CMOS Sensor , 2001 .

[37]  B. Wooley,et al.  A 128/spl times/128-pixel standard CMOS image sensor with electronic shutter , 1996, 1996 IEEE International Solid-State Circuits Conference. Digest of TEchnical Papers, ISSCC.

[38]  B. Mansoorian,et al.  A high speed, 500 frames/s, 1024/spl times/1024 CMOS active pixel sensor , 1999, 1999 Symposium on VLSI Circuits. Digest of Papers (IEEE Cat. No.99CH36326).

[39]  Bryan D. Ackland,et al.  A 200-mW, 3.3-V, CMOS Color Camera IC Producing 352 288 24-b Video at 30 Frames/s , 1998 .

[40]  Abbas El Gamal,et al.  Integration of image capture and processing: beyond single-chip digital camera , 2001, IS&T/SPIE Electronic Imaging.

[41]  A. El Gamal,et al.  CMOS image sensors , 2005, IEEE Circuits and Devices Magazine.

[42]  Alexander I. Krymski,et al.  A 9-V/Lux-s 5000-frames/s 512/spl times/512 CMOS sensor , 2003 .

[43]  A. El Gamal,et al.  A 640/spl times/512 CMOS image sensor with ultra wide dynamic range floating-point pixel-level ADC , 1999, 1999 IEEE International Solid-State Circuits Conference. Digest of Technical Papers. ISSCC. First Edition (Cat. No.99CH36278).

[44]  Chung-Yu Wu,et al.  A low-photocurrent CMOS retinal focal-plane sensor with a pseudo-BJT smoothing network and an adaptive current Schmitt trigger for scanner applications , 2004, IEEE Sensors Journal.

[45]  P. M. Acosta-Serafini,et al.  A 1/3'' VGA linear wide dynamic range CMOS image sensor implementing a predictive multiple sampling algorithm with overlapping integration intervals , 2003, IEEE Journal of Solid-State Circuits.

[46]  S. Decker,et al.  A 256/spl times/256 CMOS imaging array with wide dynamic range pixels and column-parallel digital output , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[47]  Peter Seitz.  Solid-State Image Sensing , 2000, Computer Vision and Applications.

[48]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[49]  Orly Yadid-Pecht,et al.  Empirical dark current modeling for complementary metal oxide semiconductor active pixel sensor , 2002 .

[50]  M. Ishikawa,et al.  A dynamically reconfigurable SIMD processor for a vision chip , 2003, IEEE Journal of Solid-State Circuits.

[51]  Morley M. Blouke,et al.  Charge-Coupled Devices and Solid State Optical Sensors III , 1991 .

[52]  L. Gonzo,et al.  Novel CMOS image sensor with a 132-dB dynamic range , 2002, IEEE J. Solid State Circuits.

[53]  H. Sumi,et al.  A high-sensitivity CMOS image sensor with gain-adaptive column amplifiers , 2005, IEEE Journal of Solid-State Circuits.

[54]  Xinqiao Liu,et al.  A 10000 frames/s CMOS digital pixel sensor , 2001, IEEE J. Solid State Circuits.

[55]  R. Johansson,et al.  A multiresolution 100-GOPS 4-Gpixels/s programmable smart vision sensor for multisense imaging , 2005, IEEE Journal of Solid-State Circuits.

[56]  S. Wuu,et al.  A high performance active pixel sensor with 0.18um CMOS color imager technology , 2001, International Electron Devices Meeting. Technical Digest (Cat. No.01CH37224).

[57]  T. Kurashina,et al.  A high-speed CMOS image sensor with profile data acquiring function , 2005, IEEE Journal of Solid-State Circuits.

[58]  Idaku Ishii,et al.  A digital vision chip specialized for high-speed target tracking , 2003 .

[59]  Orly Yadid-Pecht,et al.  CMOS APS with autoscaling and customized wide dynamic range , 1999 .

[60]  Gustavo Liñán Cembrano,et al.  A 1000 FPS at 128×128 vision processor with 8-bit digitized I/O , 2004, IEEE J. Solid State Circuits.