High Fill-Factor Imagers for Neuromorphic Processing Enabled by Floating-Gate Circuits

In neuromorphic modeling of the retina, it would be very nice to have processing capabilities at the focal plane while retaining the density of typical active pixel sensor (APS) imager designs. Unfortunately, these two goals have been mostly incompatible. We introduce our transform imager technology and basic architecture that uses analog floating-gate devices to make it possible to have computational imagers with high pixel densities. This imager approach allows programmable focal-plane processing that can perform retinal and higher-level bioinspired computation. The processing is performed continuously on the image via programmable matrix operations that can operate on the entire image or blocks within the image. The resulting dataflow architecture can directly perform computation of spatial transforms, motion computations, and stereo computations. The core imager performs computations at the pixel plane, but still holds a fill factor greater than 40 percent—comparable to the high fill factors of APS imagers. Each pixel is composed of a photodiode sensor element and a multiplier. We present experimental results from several imager arrays built in 0.5m process (up to in an area of 4 millimeter squared).

[1]  Krzysztof J. Cios,et al.  Advances in neural information processing systems 7 , 1997 .

[2]  H. Van Tran,et al.  A 2.5 V 256-level non-volatile analog storage device using EEPROM technology , 1996, 1996 IEEE International Solid-State Circuits Conference. Digest of TEchnical Papers, ISSCC.

[3]  Kwabena Boahen,et al.  The Retinomorphic Approach: Pixel-Parallel Adaptive Amplification, Filtering, and Quantization , 1997 .

[4]  Misha A. Mahowald,et al.  An Analog VLSI System for Stereoscopic Vision , 1994 .

[5]  P. D. Smith,et al.  A programmable diffuser circuit based on floating-gate devices , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[6]  B. A. Minch,et al.  Translinear circuits using subthreshold floating-gate MOS transistors , 1996 .

[7]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera-on-a-chip , 1997 .

[8]  Paul Hasler,et al.  Multiple-input translinear element networks , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[9]  Rahul Sarpeshkar,et al.  Efficient precise computation with noisy components: extrapolating from an electronic cochlea to the brain , 1997 .

[10]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera on a chip , 1995, Proceedings of International Electron Devices Meeting.

[11]  Bedrich J. Hosticka,et al.  Single chip CMOS image sensors for a retina implant system , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[12]  Tor Sverre Lande,et al.  Overview of floating-gate devices, circuits, and systems , 2001 .

[13]  David V. Anderson,et al.  Programmable and adaptive analog filters using arrays of floating-gate circuits , 2001, Proceedings 2001 Conference on Advanced Research in VLSI. ARVLSI 2001.

[14]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[15]  M. Kyomasu,et al.  A new MOS imager using photodiode as current source , 1991 .

[16]  Rahul Sarpeshkar,et al.  Visual Motion Computation in Analog VLSI Using Pulses , 1992, NIPS.

[17]  Orly Yadid-Pecht,et al.  Wide intrascene dynamic range CMOS APS using dual sampling , 1997 .

[18]  Ralph Etienne-Cummings,et al.  A programmable focal-plane MIMD image processor chip , 2001, IEEE J. Solid State Circuits.

[19]  Abhishek Bandyopadhyay,et al.  A matrix transform imager allowing high-fill factor , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[20]  Tobi Delbrück,et al.  Silicon retina with correlation-based, velocity-tuned pixels , 1993, IEEE Trans. Neural Networks.

[21]  Trevor Blyth,et al.  FP 16.6: A 2.5V 256-Level Non-Volatile Analog Storage Device Using EEPROM Technology , 1995 .

[22]  Gealow,et al.  A Pixel-parallel Image Processor Using Logic Pitch-matched To Dynamic Memory , 1997, Symposium 1997 on VLSI Circuits.

[23]  Paul E. Hasler,et al.  Floating-gate devices, circuits, and systems , 2005, Fifth International Workshop on System-on-Chip for Real-Time Applications (IWSOC'05).

[24]  Andreas G. Andreou,et al.  A Contrast Sensitive Silicon Retina with Reciprocal Synapses , 1991, NIPS.

[25]  E.R. Fossum,et al.  A 1.2 V micropower CMOS active pixel image sensor for portable applications , 2000, 2000 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.00CH37056).

[26]  Ran Ginosar,et al.  A random access photodiode array for intelligent image capture , 1991 .

[27]  Ralph Etienne-Cummings,et al.  Implementation of steerable spatiotemporal image filters on the focal plane , 2002 .

[28]  Paul Hasler,et al.  Offset removal from floating gate differential amplifiers and mixers , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[29]  Tobi Delbrück,et al.  An Electronic Photoreceptor Sensitive to Small Changes in Intensity , 1988, NIPS.

[30]  T. Delbruck.  Silicon retina with correlation-based, velocity-tuned pixels , 1993 .

[31]  Carver Mead.  Scaling of MOS technology to submicrometer feature sizes , 1994, J. VLSI Signal Process..

[32]  Gert Cauwenberghs,et al.  Floating-gate adaptation for focal-plane online nonuniformity correction , 2001 .

[33]  Misha Mahowald,et al.  Analog VLSI chip for stereocorrespondence , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[34]  Carver A. Mead,et al.  Time-derivative adaptive silicon photoreceptor array , 1991, Optics & Photonics.

[35]  Paul Hasler,et al.  Biologically inspired auditory sensing system interfaces on a chip , 2002, Proceedings of IEEE Sensors.

[36]  Eric R. Fossum,et al.  Digital camera system on a chip , 1998, IEEE Micro.

[37]  Carver A. Mead,et al.  An Integrated Analog Optical Motion Sensor , 1986 .

[38]  Christof Koch,et al.  A Robust Analog VLSI Reichardt Motion Sensor , 2000 .

[39]  Christof Koch,et al.  An Analog VLSI Inplementation of a Visual Interneuron Enhanced Sensory Processing Through Biophysical Modeling , 1999, Int. J. Neural Syst..

[40]  Paul E. Hasler,et al.  Single Transistor Learning Synapses , 1994, NIPS.

[41]  Tonia Morris,et al.  A column-based processing array for high-speed digital image processing , 1999, Proceedings 20th Anniversary Conference on Advanced Research in VLSI.

[42]  Reid R. Harrison,et al.  A CMOS programmable analog memory-cell array using floating-gate circuits , 2001 .

[43]  C. Mead,et al.  Neuromorphic Robot Vision with Mixed Analog- Digital Architecture , 2005 .

[44]  Paul E. Hasler,et al.  Accurate programming of analog floating-gate arrays , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[45]  Christof Koch,et al.  A Modular Multi-Chip Neuromorphic Architecture for Real-Time Visual Motion Processing , 2000 .

[46]  S. Decker,et al.  A 256/spl times/256 CMOS imaging array with wide dynamic range pixels and column-parallel digital output , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[47]  Tadashi Shibata,et al.  A functional MOS transistor featuring gate-level weighted sum and threshold operations , 1992 .

[48]  Kwabena Boahen,et al.  A throughput-on-demand address-event transmitter for neuromorphic chips , 1999, Proceedings 20th Anniversary Conference on Advanced Research in VLSI.

[49]  Paul E. Hasler,et al.  Adaptive circuits using pFET floating-gate devices , 1999, Proceedings 20th Anniversary Conference on Advanced Research in VLSI.

[50]  Stephen P. DeWeerth,et al.  A CMOS imager with real-time frame differencing and centroid computation , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[51]  David V. Anderson,et al.  Cooperative analog-digital signal processing , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.