Review of bioinspired real-time motion analysis systems

Flying insects are able to manoeuvre through complex environments with remarkable ease and accuracy despite their simple visual system. Physiological evidence suggests that flight control is primarily guided by a small system of neurons tuned to very specific types of complex motion. This system is a promising model for bio-inspired approaches to low-cost artificial motion analysis systems, such as collision avoidance devices. A number of models of motion detection have been proposed, with the basic model being the Reichardt Correlator. Electrophysiological data suggest a variety of non-linear elaborations, which include compressive non-linearities and adaptive feedback of local motion detector outputs. In this paper we review a number of computational models for motion detection from the point of view of ease of implementation in low cost VLSI technology. We summarise the features of biological motion analysis systems that are important for the design of real-time artificial motion analysis systems. Then we report on recent progress in bio-inspired analog VLSI chips that capture properties of biological neural computation.

[1]  E. Buchner.  Elementary movement detectors in an insect visual system , 1976, Biological Cybernetics.

[2]  Christof Koch,et al.  An Analog VLSI Chip for Finding Edges from Zero-crossings , 1990, NIPS.

[3]  Jörg Kramer.  Compact Integrated Motion Sensor With Three-Pixel Interaction , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Massimo Gottardi,et al.  A CCD/CMOS image motion sensor , 1993, 1993 IEEE International Solid-State Circuits Conference Digest of Technical Papers.

[5]  John Lazzaro,et al.  A Delay-Line Based Motion Detection Chip , 1990, NIPS.

[6]  Erhan Ozalevli,et al.  Reconfigurable biologically inspired visual motion systems using modular neuromorphic VLSI chips , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[7]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[8]  E. Vittoz,et al.  Analog Storage of Adjustable Synaptic Weights , 1991 .

[9]  Giacomo Indiveri,et al.  Parallel analog VLSI architectures for computation of heading direction and time-to-contact , 1995, NIPS.

[10]  Kiyoharu Aizawa,et al.  Motion-adaptive image sensor for enhancement and wide dynamic range , 1996, Advanced Imaging and Network Technologies.

[11]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[12]  Rahul Sarpeshkar,et al.  Visual Motion Computation in Analog VLSI Using Pulses , 1992, NIPS.

[13]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[14]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[15]  Giacomo Indiveri,et al.  Analog VLSI architecture for computing heading direction , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[16]  Charles M. Higgins,et al.  A biomimetic VLSI architecture for small target tracking , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[17]  J.L. Huertas,et al.  Switched-current techniques for image processing cellular neural networks in MOS VLSI , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[18]  W. Reichardt,et al.  Computational structure of a biological motion-detection system as revealed by local detector analysis in the fly's nervous system. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[19]  John Lazzaro,et al.  Computing motion using analog VLSI vision chips: An experimental comparison among different approaches , 1992, International Journal of Computer Vision.

[20]  S. Espejo,et al.  Cellular neural network chips with optical image acquisition , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[21]  Kiyoharu Aizawa,et al.  A novel image sensor for video compression , 1994, Proceedings of 1st International Conference on Image Processing.

[22]  A. Rodriguez-Vazquez,et al.  A continuous-time cellular neural network chip for direction-selectable connected component detection with optical image acquisition , 1994, Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems.

[23]  Alexa Riehle,et al.  Directionally Selective Motion Detection by Insect Neurons , 1989 .

[24]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[25]  Richard F. Lyon,et al.  The Optical Mouse, and an Architectural Methodology for Smart Digital Sensors , 1981 .

[26]  Hans-Hellmut Nagel,et al.  Displacement vectors derived from second-order intensity variations in image sequences , 1983, Comput. Vis. Graph. Image Process..

[27]  R. Haralick,et al.  The Facet Approach to Optic Flow , 1983 .

[28]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[29]  R. Etienne-Cummings,et al.  A new temporal domain optical flow measurement technique for focal plane VLSI implementation , 1993, 1993 Computer Architectures for Machine Perception.

[30]  Ángel Rodríguez-Vázquez,et al.  CMOS optical-sensor array with high output current levels and automatic signal-range centring , 1994 .

[31]  Alessandro Verri,et al.  Computing optical flow from an overconstrained system of linear algebraic equations , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[32]  J. Kelsey.  Studies on Excitation and Inhibition in the Retina , 1976 .

[33]  I.S. McQuirk,et al.  An analog VLSI chip for estimating the focus of expansion , 1997, 1997 IEEE International Solids-State Circuits Conference. Digest of Technical Papers.

[34]  X. Arreguit,et al.  A CMOS motion detector system for pointing devices , 1996, 1996 IEEE International Solid-State Circuits Conference. Digest of TEchnical Papers, ISSCC.

[35]  Larry S. Davis,et al.  Motion estimation based on multiple local constraints and nonlinear smoothing , 1983, Pattern Recognit..

[36]  M. Egelhaaf,et al.  Processing of figure and background motion in the visual system of the fly , 1989, Biological Cybernetics.

[37]  Robert B. Pinter,et al.  Image Motion Processing in Biological and Computer Vision Systems , 1989, Other Conferences.

[38]  Christof Koch,et al.  Compact real-time 2D gradient-based analog VLSI motion sensor , 1998, Other Conferences.

[39]  K. Kirschfeld.  The visual system of Musca: Studies on optics, structure and function , 1972 .

[40]  Christof Koch,et al.  A Robust Analog VLSI Motion Sensor Based on the Visual System of the Fly , 1999, Auton. Robots.

[41]  Abdesselam Bouzerdoum,et al.  A simple model of the SUSTAINED unit in the insect lamina , 1996, 1996 Australian New Zealand Conference on Intelligent Information Systems. Proceedings. ANZIIS 96.

[42]  Derek Abbott,et al.  An analog implementation of early visual processing in insects , 1993, 1993 International Symposium on VLSI Technology, Systems, and Applications Proceedings of Technical Papers.

[43]  H. Jiang,et al.  A 2-D velocity- and direction-selective sensor with BJT-based silicon retina and temporal zero-crossing detector , 1999 .

[44]  Juyang Weng,et al.  A theory of image matching , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[45]  C. Cafforio,et al.  Tracking moving objects in television images , 1979 .

[46]  Giacomo Indiveri,et al.  Analog VLSI architectures for motion processing: from fundamental limits to system applications , 1996, Proc. IEEE.

[47]  Alexander Borst,et al.  Principles of visual motion detection , 1989, Trends in Neurosciences.

[48]  Werner Reichardt,et al.  Figure-ground discrimination by relative movement in the visual system of the fly , 2004, Biological Cybernetics.

[49]  L. Dron.  The multiscale veto model: A two-stage analog network for edge detection and image reconstruction , 1993 .

[50]  Rahul Sarpeshkar,et al.  An analog VLSI velocity sensor , 1995, Proceedings of ISCAS'95 - International Symposium on Circuits and Systems.

[51]  Andreas G. Andreou,et al.  Silicon retina for motion computation , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[52]  Ralph Etienne-Cummings,et al.  Hardware implementation of a visual-motion pixel using oriented spatiotemporal neural filters , 1999 .

[53]  David J. Fleet.  Measurement of image velocity , 1992 .

[54]  Jan Van der Spiegel,et al.  A focal plane visual motion measurement sensor , 1997 .

[55]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[56]  Kiyoharu Aizawa,et al.  An image sensor for on-sensor-compression , 1995, Proceedings of Conference on Computer Architectures for Machine Perception.

[57]  Ellen C Hildreth,et al.  Computing the velocity field along contours , 1986, Workshop on Motion.

[58]  Alessandro Verri,et al.  Against Quantitative Optical Flow , 1987 .

[59]  Andreas G. Andreou,et al.  A Contrast Sensitive Silicon Retina with Reciprocal Synapses , 1991, NIPS.

[60]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[61]  G A Horridge,et al.  A template theory to relate visual processing to digital circuitry , 1990, Proceedings of the Royal Society of London. B. Biological Sciences.

[62]  Tobi Delbrück.  A Chip that Focuses an Image on Itself , 1989 .

[63]  G. Horridge.  The evolution of visual processing and the construction of seeing systems , 1987, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[64]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[65]  Andreas G. Andreou,et al.  A sampled-data motion chip , 1993, Proceedings of 36th Midwest Symposium on Circuits and Systems.

[66]  Alan A. Stocker,et al.  Analog VLSI focal-plane array with dynamic connections for the estimation of piecewise-smooth optical flow , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[67]  Christof Koch,et al.  Multiplication-based analog motion detection chip , 1991, Defense, Security, and Sensing.

[68]  L. D. McIlrath.  A CCD/CMOS focal-plane array edge detection processor implementing the multiscale veto algorithm , 1996 .

[69]  Reid R. Harrison,et al.  A CMOS Imager with On-Chip Temporal Filtering for Motion Pre-Processing , 2002 .

[70]  Rainer A. Deutschmann,et al.  Pulse-based 2d Motion Sensors Pulse-based 2d Motion Sensors , 1999 .

[71]  B. Hassenstein,et al.  Systemtheoretische Analyse der Zeit-, Reihenfolgen- und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers Chlorophanus , 1956 .

[72]  Carver A. Mead,et al.  An Integrated Analog Optical Motion Sensor , 1986 .

[73]  Rahul Sarpeshkar,et al.  Pulse-Based Analog VLSI Velocity Sensors , 1997 .

[74]  Christof Koch,et al.  A Robust Analog VLSI Reichardt Motion Sensor , 2000 .

[75]  H. Barlow,et al.  The mechanism of directionally selective units in rabbit's retina. , 1965, The Journal of physiology.

[76]  R. B. Pinter.  Adaptation of receptive field spatial organization via multiplicative lateral inhibition. , 1984, Journal of theoretical biology.

[77]  Tobi Delbrück,et al.  Silicon retina with correlation-based, velocity-tuned pixels , 1993, IEEE Trans. Neural Networks.

[78]  H. Bülthoff,et al.  Using neuropharmacology to distinguish between excitatory and inhibitory movement detection mechanisms in the fly Calliphora erythrocephala , 1988, Biological Cybernetics.

[79]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[80]  Christof Koch,et al.  An Analog VLSI Inplementation of a Visual Interneuron Enhanced Sensory Processing Through Biophysical Modeling , 1999, Int. J. Neural Syst..

[81]  Kiyoharu Aizawa,et al.  Focal plane compression and enhancement sensors , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[82]  N. Strausfeld,et al.  Visual motion detection circuits in flies: peripheral motion computation by identified small-field retinotopic neurons , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[83]  Martin Egelhaaf,et al.  On the neuronal basis of figure-ground discrimination by relative motion in the visual system of the fly , 1985, Biological Cybernetics.

[84]  Kamran Eshraghian,et al.  Two-dimensional motion detector based on insect vision , 1996, Advanced Imaging and Network Technologies.

[85]  Vasilis Z. Marmarelis,et al.  An artificial neural network for motion detection and speed estimation , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[86]  Peter J. Sobey.  Determining range information from self-motion: the template model , 1991, Other Conferences.

[87]  Peter J. Sobey,et al.  An artificial seeing system copying insect vision , 1991 .