Robust Models for Optic Flow Coding in Natural Scenes Inspired by Insect Biology

The extraction of accurate self-motion information from the visual world is a difficult problem that has been solved very efficiently by biological organisms utilizing non-linear processing. Previous bio-inspired models for motion detection based on a correlation mechanism have been dogged by issues that arise from their sensitivity to undesired properties of the image, such as contrast, which vary widely between images. Here we present a model with multiple levels of non-linear dynamic adaptive components based directly on the known or suspected responses of neurons within the visual motion pathway of the fly brain. By testing the model under realistic high-dynamic range conditions we show that the addition of these elements makes the motion detection model robust across a large variety of images, velocities and accelerations. Furthermore the performance of the entire system is more than the incremental improvements offered by the individual components, indicating beneficial non-linear interactions between processing stages. The algorithms underlying the model can be implemented in either digital or analog hardware, including neuromorphic analog VLSI, but defy an analytical solution due to their dynamic non-linear operation. The successful application of this algorithm has applications in the development of miniature autonomous systems in defense and civilian roles, including robotics, miniature unmanned aerial vehicles and collision avoidance sensors.

[1]  T. Collett,et al.  Visual control of flight behaviour in the hoverflySyritta pipiens L. , 1975, Journal of comparative physiology.

[2]  A. Straw,et al.  A `bright zone' in male hoverfly (Eristalis tenax) eyes and associated faster motion detection and increased contrast sensitivity , 2006, Journal of Experimental Biology.

[3]  M. V. Srinivasan,et al.  The waterfall illusion in an insect visual system , 1979, Vision Research.

[4]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[5]  Hateren,et al.  Blowfly flight and optic flow. I. Thorax kinematics and flight dynamics , 1999, The Journal of experimental biology.

[6]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[7]  M Egelhaaf,et al.  Are there separate ON and OFF channels in fly motion vision? , 1992, Visual Neuroscience.

[8]  A. Dubs The spatial integration of signals in the retina and lamina of the fly compound eye under different conditions of luminance , 1982, Journal of comparative physiology.

[9]  N. Franceschini,et al.  Motion detection in flies: Parametric control over ON-OFF pathways , 2004, Experimental Brain Research.

[10]  N. Strausfeld,et al.  Some Quantitative Aspects of the Fly’s Brain , 1976 .

[11]  Silvia Silva da Costa Botelho,et al.  Vision-Based Motion Detection Using C-NLPCA Approach , 2006, 2006 Ninth Brazilian Symposium on Neural Networks (SBRN'06).

[12]  N. Strausfeld,et al.  Dissection of the Peripheral Motion Channel in the Visual System of Drosophila melanogaster , 2007, Neuron.

[13]  H. Krapp,et al.  Visuomotor Transformation in the Fly Gaze Stabilization System , 2008, PLoS biology.

[14]  David S. Williams Changes of photoreceptor performance associated with the daily turnover of photoreceptor membrane in locusts , 1983, Journal of comparative physiology.

[15]  Ajit Singh,et al.  Incremental estimation of image flow using a Kalman filter , 1992, J. Vis. Commun. Image Represent..

[16]  Said F. Al-Sarawi,et al.  Smart Structures, Devices, and Systems IV , 2007 .

[17]  M. F. Land,et al.  Maps of the acute zones of fly eyes , 1985, Journal of Comparative Physiology A.

[18]  E. Van der Burg,et al.  Audiovisual events capture attention: evidence from temporal order judgments. , 2008, Journal of vision.

[19]  J. H. Van Hateren,et al.  Directional tuning curves, elementary movement detectors, and the estimation of the direction of visual movement , 1990, Vision Research.

[20]  A Borst,et al.  Fly motion vision is based on Reichardt detectors regardless of the signal-to-noise ratio. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Martin Egelhaaf,et al.  INTRINSIC PROPERTIES OF BIOLOGICAL MOTION DETECTORS PREVENT THE OPTOMOTOR CONTROL SYSTEM FROM GETTING UNSTABLE , 1996 .

[22]  M V Srinivasan,et al.  Honeybee navigation: nature and calibration of the "odometer". , 2000, Science.

[23]  J. H. van Hateren,et al.  A theory of maximizing sensory information , 2004, Biological Cybernetics.

[24]  Alexander Borst,et al.  Correlation versus gradient type motion detectors: the pros and cons , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[25]  Russell S. A. Brinkworth,et al.  Characterization of a neuromorphic motion detection chip based on insect visual system , 2009, 2009 International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

[26]  D. G. Stavenga,et al.  Spectral sensitivity of blowfly photoreceptors: Dependence on waveguide effects and pigment concentration , 1986, Vision Research.

[27]  Hans-Hellmut Nagel,et al.  On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results , 1987, Artif. Intell..

[28]  N. Strausfeld,et al.  Visual motion detection circuits in flies: peripheral motion computation by identified small-field retinotopic neurons , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  Simon B. Laughlin,et al.  The Role of Natural Image Statistics in Biological Motion Estimation , 2000, Biologically Motivated Computer Vision.

[30]  D. Stavenga Angular and spectral sensitivity of fly photoreceptors. I. Integrated facet lens and rhabdomere optics , 2002, Journal of Comparative Physiology A.

[31]  N. Franceschini,et al.  From insect vision to robot vision , 1992 .

[32]  Derek Abbott,et al.  Effect of spatial sampling on pattern noise in insect-based motion detection , 2005, SPIE Micro + Nano Materials, Devices, and Applications.

[33]  S. Laughlin,et al.  Adaptation of the motion-sensitive neuron H1 is generated locally and governed by contrast frequency , 1985, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[34]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[35]  Robert A. Harris,et al.  Contrast Gain Reduction in Fly Motion Adaptation , 2000, Neuron.

[36]  A. Borst,et al.  A look into the cockpit of the fly: visual orientation, algorithms, and identified neurons , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  J. van Santen,et al.  Temporal covariance model of human motion perception. , 1984, Journal of the Optical Society of America. A, Optics and image science.

[38]  Wolfram Burgard,et al.  Monte Carlo localization for mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[39]  J. Maunsell,et al.  The effects of parvocellular lateral geniculate lesions on the acuity and contrast sensitivity of macaque monkeys , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  David C. O'Carroll,et al.  Implementation of an elaborated neuromorphic model of a biological photoreceptor , 2008, Biological Cybernetics.

[41]  R. Hengstenberg,et al.  Estimation of self-motion by optic flow processing in single visual interneurons , 1996, Nature.

[42]  Paul D. Barnett,et al.  Sexual Dimorphism in the Hoverfly Motion Vision Pathway , 2008, Current Biology.

[43]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[44]  Derek Abbott,et al.  Implementation of saturation for modelling pattern noise using naturalistic stimuli , 2006, SPIE Micro + Nano Materials, Devices, and Applications.

[45]  K. Kirschfeld Aufnahme und Verarbeitung optischer Daten im Komplexauge von Insekten , 1971, Naturwissenschaften.

[46]  M. Srinivasan,et al.  Spectral properties of movement perception in the dronefly Eristalis , 2004, Journal of Comparative Physiology A.

[47]  D. Tolhurst,et al.  Amplitude spectra of natural images. , 1992, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[48]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[49]  H. Honegger Receptive fields of sustained medulla neurons in crickets , 1980, Journal of comparative physiology.

[50]  M. Ibbotson Evidence for velocity–tuned motion-sensitive descending neurons in the honeybee , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[51]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[52]  Allan W. Snyder,et al.  Spatial information capacity of compound eyes , 2004, Journal of comparative physiology.

[53]  M. Egelhaaf,et al.  Vision in flying insects , 2002, Current Opinion in Neurobiology.

[54]  W Reichardt,et al.  Autocorrelation, a principle for evaluation of sensory information by the central nervous system , 1961 .

[55]  W. Reichardt,et al.  Properties of individual movement detectors as derived from behavioural experiments on the visual system of the fly , 1988, Biological Cybernetics.

[56]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[57]  M. Land Visual acuity in insects. , 1997, Annual review of entomology.

[58]  R. A Harris,et al.  Afterimages in fly motion vision , 2002, Vision Research.

[59]  Hugh L. Kennedy Gradient Operators for the Determination of Optical Flow , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).

[60]  A. Borst,et al.  Transient and steady-state response properties of movement detectors. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[61]  B. Hassenstein,et al.  Systemtheoretische Analyse der Zeit-, Reihenfolgen- und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers Chlorophanus , 1956 .

[62]  Christof Koch,et al.  A Robust Analog VLSI Reichardt Motion Sensor , 2000 .

[63]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[64]  R O Dror,et al.  Accuracy of velocity estimation by Reichardt correlators. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[65]  H. P. Snippe,et al.  Phototransduction in primate cones and blowfly photoreceptors: different mechanisms, different algorithms, similar response , 2005, Journal of Comparative Physiology A.

[66]  W. Reichardt,et al.  Computational structure of a biological motion-detection system as revealed by local detector analysis in the fly's nervous system. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[67]  J. H. Hateren,et al.  Information theoretical evaluation of parametric models of gain control in blowfly photoreceptor cells , 2001, Vision Research.

[68]  T. Collett,et al.  Chasing behaviour of houseflies (Fannia canicularis) , 1974, Journal of comparative physiology.

[69]  Heiko Neumann,et al.  A Fast Biologically Inspired Algorithm for Recurrent Motion Estimation , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Eng-Leng Mah,et al.  Photoreceptor processing improves salience facilitating small target detection in cluttered scenes. , 2008, Journal of vision.

[71]  J. P. Lindemann,et al.  Function of a Fly Motion-Sensitive Neuron Matches Eye Movements during Free Flight , 2005, PLoS biology.

[72]  David C. O'Carroll,et al.  Performance of a bio-inspired model for the robust detection of moving targets in high dynamic range natural scenes , 2010 .

[73]  W. Bialek,et al.  Statistical mechanics and visual signal processing , 1994, cond-mat/9401072.

[74]  K. Kirschfeld,et al.  Motion sensitivity in the nucleus of the basal optic root of the pigeon. , 1994, Journal of neurophysiology.

[75]  Patrick A. Shoemaker,et al.  A Model for the Detection of Moving Targets in Visual Clutter Inspired by Insect Physiology , 2008, PloS one.

[76]  David J. Fleet,et al.  Stability of Phase Information , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[77]  R. B. Pinter,et al.  Nonlinear Vision: Determination of Neural Receptive Fields, Function, and Networks , 1992 .

[78]  R. Olberg,et al.  Prey pursuit and interception in dragonflies , 2000, Journal of Comparative Physiology A.

[79]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[80]  Alexander Borst,et al.  Principles of visual motion detection , 1989, Trends in Neurosciences.

[81]  J. H. van Hateren,et al.  Pattern recognition in bees: orientation discrimination , 1990, Journal of Comparative Physiology A.

[82]  Daniel Osorio,et al.  Mechanisms for Neural Signal Enhancement in the Blowfly Compound Eye , 1989 .

[83]  S. B. Laughlin,et al.  Angular sensitivity of the retinula cells of dark-adapted worker bee , 1971, Zeitschrift für vergleichende Physiologie.

[84]  R. G. Guy,et al.  Are the large monopolar cells of the insect lamina on the optomotor pathway? , 2004, Journal of Comparative Physiology A.

[85]  A. Straw,et al.  Contrast sensitivity of insect motion detectors to natural images. , 2008, Journal of vision.

[86]  D. Tolhurst,et al.  Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes , 2000, Vision Research.

[87]  W. Reichardt,et al.  Elementary pattern discrimination (behavioural experiments with the fly Musca domestica) , 1986, Biological Cybernetics.

[88]  Zhang,et al.  Honeybee navigation en route to the goal: visual flight control and odometry , 1996, The Journal of experimental biology.

[89]  Robert A. Harris,et al.  Adaptation and the temporal delay filter of fly motion detectors , 1999, Vision Research.

[90]  M Egelhaaf,et al.  On the Computations Analyzing Natural Optic Flow: Quantitative Model Analysis of the Blowfly Motion Vision Pathway , 2005, The Journal of Neuroscience.

[91]  D. Osorio Mechanisms of early visual processing in the medulla of the locust optic lobe: How self-inhibition, spatial-pooling, and signal rectification contribute to the properties of transient cells , 1991, Visual Neuroscience.

[92]  Patrick A. Shoemaker,et al.  Velocity constancy and models for wide-field visual motion detection in insects , 2005, Biological Cybernetics.

[93]  Zuley Rivera-Alvidrez,et al.  Contrast saturation in a neuronally-based model of elementary motion detection , 2005, Neurocomputing.

[94]  S. Laughlin,et al.  Spatio-temporal properties of motion detectors matched to low image velocities in hovering insects , 1997, Vision Research.

[95]  Roger C. Hardie,et al.  Common strategies for light adaptation in the peripheral visual systems of fly and dragonfly , 1978, Journal of comparative physiology.

[96]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[97]  Alexander Borst,et al.  Mechanisms of dendritic integration underlying gain control in fly motion-sensitive interneurons , 1995, Journal of Computational Neuroscience.

[98]  R. O. Uusitalo,et al.  Transfer of graded potentials at the photoreceptor-interneuron synapse , 1995, The Journal of general physiology.

[99]  Allan W. Snyder,et al.  Acuity of compound eyes: Physical limitations and design , 2004, Journal of comparative physiology.

[100]  C. W. G Clifford,et al.  Fundamental mechanisms of visual motion detection: models, cells and functions , 2002, Progress in Neurobiology.

[101]  N. Strausfeld,et al.  Neuronal basis for parallel visual processing in the fly , 1991, Visual Neuroscience.

[102]  D. Osorio,et al.  The temporal properties of non-linear, transient cells in the locust medulla , 2004, Journal of Comparative Physiology A.

[103]  S. Laughlin,et al.  Insect motion detectors matched to visual ecology , 1996, Nature.

[104]  Patrick A. Shoemaker,et al.  Bio-inspired optical rotation sensor , 2006, SPIE Micro + Nano Materials, Devices, and Applications.