Sensory computing

Computation in artificial perceptual systems assumes that appropriate and reliable sensory information about the environment is available. However, today's sensors cannot guarantee optimal information at all times. For example, when an image from a CCD camera saturates, the entire vision system fails regardless of how 'algorithmically' sophisticated it is. The principal goal of sensory computing is to extract useful information about the environment from 'imperfect' sensors. This paper attempts to generalize our experience with smart vision sensors and provide a direction and illustration for exploiting complex spatio-temporal interaction of image formation, signal detectors, and on-chip processing to extract a surprising amount of useful information from on-chip systems. The examples presented include: VLSI sensory computing systems for adaptive imaging, ultra fast feature tracking with attention, and ultra fast range imaging. Using these examples, we illustrate how sensory computing can extract unique, rich and otherwise not obtainable sensory information when an appropriate balance is maintained between sensing modality, algorithms and available technology.

[1]  T. G. Morris,et al.  Analog VLSI circuits for covert attentional shifts , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[2]  B.J. Hosticka,et al.  A high dynamic range CMOS image sensor for automotive applications , 2000, Proceedings of the 25th European Solid-State Circuits Conference.

[3]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[4]  Christof Koch,et al.  Visual Information Processing: From Neurons to Chips , 1991 .

[5]  G. P. Weckler,et al.  Operation of p-n Junction Photodetectors in a Photon Flux Integrating Mode , 1967 .

[6]  Keiichi Yamada,et al.  Effectiveness of video camera dynamic range expansion for lane mark detection , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[7]  B. Schneider,et al.  Vertically integrated sensors for advanced imaging applications , 2000, IEEE Journal of Solid-State Circuits.

[8]  Vladimir Brajovic,et al.  Computational Sensors for Global Operations in Vision , 1996 .

[9]  M. Ishikawa,et al.  A CMOS vision chip with SIMD processing element array for 1 ms image processing , 1999, 1999 IEEE International Solid-State Circuits Conference. Digest of Technical Papers. ISSCC. First Edition (Cat. No.99CH36278).

[10]  Yukio Sato,et al.  High Speed Rangefinder , 1988, Other Conferences.

[11]  Christof Koch,et al.  Analog VLSI Circuits for Attention-Based, Visual Tracking , 1996, NIPS.

[12]  Takeo Kanade,et al.  Temporal photoreception for adaptive dynamic range image sensing and encoding , 1998, Neural Networks.

[13]  Andreas G. Andreou,et al.  Current-mode subthreshold MOS circuits for analog VLSI neural systems , 1991, IEEE Trans. Neural Networks.

[14]  Takeo Kanade,et al.  Integrated sensor and range-finding analog signal processor , 1991 .

[15]  John Lazzaro,et al.  Winner-Take-All Networks of O(N) Complexity , 1988, NIPS.

[16]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[17]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[18]  Takeo Kanade,et al.  A sorting image sensor: an example of massively parallel intensity-to-time processing for low-latency computational sensors , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[19]  E. Fossum,et al.  CMOS active pixel image sensors for highly integrated imaging systems , 1997, IEEE J. Solid State Circuits.

[20]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[21]  Alireza Moini,et al.  Vision Chips , 1999 .