A Vision Chip for Color Segmentation and Pattern Matching

A 128(H) 64(V) RGB CMOS imager is integrated with region-of-interest selection, RGB-to-HSI transformation, HSI-based pixel segmentation, (36bins 12bits)-HSI histogramming, and sum-of-absolute-difference (SAD) template matching. Thirty-two learned color templates are stored and compared to each image. The chip captures the R, G, and B images using in-pixel storage before passing the pixel content to a multiplying digital-to-analog converter (DAC) for white balancing. The DAC can also be used to pipe in images for a PC. The color processing uses a biologically inspired color opponent representation and an analog lookup table to determine the Hue (H) of each pixel. Saturation (S) is computed using a loser-take-all circuit. Intensity (I) is given by the sum of the color components. A histogram of the segments of the image, constructed by counting the number of pixels falling into 36 Hue intervals of 10 degrees, is stored on a chip and compared against the histograms of new segments using SAD comparisons. We demonstrate color-based image segmentation and object recognition with this chip. Running at 30 fps, it uses 1 mW. To our knowledge, this is the first chip that integrates imaging, color segmentation, and color-based object recognition at the focal plane.

[1]  F. Werblin,et al.  Control of Retinal Sensitivity: I. Light and Dark Adaptation of Vertebrate Rods and Cones , 1974 .

[2]  F. Werblin.  Control of Retinal Sensitivity II . Lateral Interactions at the Outer Plexiform Layer , 2022 .

[3]  F. Werblin,et al.  I. Light and Dark Adaptation of Vertebrate Rods and Cones , 1974 .

[4]  Carver A. Mead.  A Sensitive Electronic Photoreceptor , 1985 .

[5]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[6]  John Lazzaro,et al.  A Silicon Model Of Auditory Localization , 1989, Neural Computation.

[7]  Eric A. Vittoz,et al.  Analog VLSI implementation of neural networks , 1990, IEEE International Symposium on Circuits and Systems.

[8]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[9]  Rodney M. Goodman,et al.  A real-time neural system for color constancy , 1991, IEEE Trans. Neural Networks.

[10]  Ran Ginosar,et al.  A random access photodiode array for intelligent image capture , 1991 .

[11]  Charles Sodini,et al.  Switched capacitor networks for focal plane image processing systems , 1992, IEEE Trans. Circuits Syst. Video Technol..

[12]  Sunetra K. Mendis,et al.  A 128/spl times/128 CMOS active pixel image sensor for highly integrated imaging systems , 1993, Proceedings of IEEE International Electron Devices Meeting.

[13]  Sunetra K. Mendis,et al.  A 128 x 128 CMOS Active Pixel Image Sensor for Highly Integrated Imaging Systems , 1993 .

[14]  Christof Koch,et al.  Vision Chips: Implementing Vision Algorithms with Analog VLSI Circuits , 1994 .

[15]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera on a chip , 1995, Proceedings of International Electron Devices Meeting.

[16]  A. Dickinson,et al.  Camera on a chip , 1996, 1996 IEEE International Solid-State Circuits Conference. Digest of TEchnical Papers, ISSCC.

[17]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[18]  K. Azadet,et al.  A 200 mW 3.3 V CMOS color camera IC producing 352/spl times/288 24 b video at 30 frames/s , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[19]  Takeo Kanade,et al.  Computational sensor for visual tracking with attention , 1998, IEEE J. Solid State Circuits.

[20]  Bryan D. Ackland,et al.  A 200-mW, 3.3-V, CMOS Color Camera IC Producing 352 288 24-b Video at 30 Frames/s , 1998 .

[21]  Tamás Roska,et al.  20 msec focal plane image processing , 2000 .

[22]  Tamás Roska,et al.  20 /spl mu/sec focal plane image processing , 2000, Proceedings of the 2000 6th IEEE International Workshop on Cellular Neural Networks and their Applications (CNNA 2000) (Cat. No.00TH8509).

[23]  E. Rolls.  Functions of the Primate Temporal Lobe Cortical Visual Areas in Invariant Visual Object and Face Recognition , 2000, Neuron.

[24]  Ralph Etienne-Cummings,et al.  A programmable focal-plane MIMD image processor chip , 2001 .

[25]  Giacomo Indiveri,et al.  A Current-Mode Hysteretic Winner-take-all Network, with Excitatory and Inhibitory Coupling , 2001 .

[26]  Kwangho Yoon,et al.  Single-chip CMOS image sensor for mobile applications , 2002, 2002 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.02CH37315).

[27]  Ralph Etienne-Cummings,et al.  Implementation of steerable spatiotemporal image filters on the focal plane , 2002 .

[28]  Ralph Etienne-Cummings.  Biologically Inspired Visual Motion Detection in VLSI , 2004, International Journal of Computer Vision.

[29]  Christof Koch,et al.  Toward color image segmentation in analog VLSI: Algorithm and hardware , 1994, International Journal of Computer Vision.