FLIVVER: Fly Lobula Inspired Visual Velocity Estimation & Ranging

The mechanism by which a tiny insect or insect-sized robot could estimate its absolute velocity and distance to nearby objects remains unknown. However, this ability is critical for behaviors that require estimating wind direction during flight, such as odor-plume tracking. Neuroscience and behavior studies with insects have shown that they rely on the perception of image motion, or optic flow, to estimate relative motion, equivalent to a ratio of their velocity and distance to objects in the world. The key open challenge is therefore to decouple these two states from a single measurement of their ratio. Although modern SLAM (Simultaneous Localization and Mapping) methods provide a solution to this problem for robotic systems, these methods typically rely on computations that insects likely cannot perform, such as simultaneously tracking multiple individual visual features, remembering a 3D map of the world, and solving nonlinear optimization problems using iterative algorithms. Here we present a novel algorithm, FLIVVER, which combines the geometry of dynamic forward motion with inspiration from insect visual processing to \textit{directly} estimate absolute ground velocity from a combination of optic flow and acceleration information. Our algorithm provides a clear hypothesis for how insects might estimate absolute velocity, and also provides a theoretical framework for designing fast analog circuitry for efficient state estimation, which could be applied to insect-sized robots.

[1]  Christopher E. Neely,et al.  Mixed-mode VLSI optic flow sensors for in-flight control of a micro air vehicle , 2000, SPIE Optics + Photonics.

[2]  William T. Freeman,et al.  Learning the Depths of Moving People by Watching Frozen People , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Rui Fang,et al.  Robust Stereo Visual-Inertial Odometry Using Nonlinear Optimization , 2019, Sensors.

[4]  Stephen P. Boyd,et al.  CVXPY: A Python-Embedded Modeling Language for Convex Optimization , 2016, J. Mach. Learn. Res..

[5]  L. R. Carley,et al.  An analog circuit technique for finding the median , 1993, Proceedings of IEEE Custom Integrated Circuits Conference - CICC '93.

[6]  Mau-Chung Frank Chang,et al.  An Analog Neural Network Computing Engine Using CMOS-Compatible Charge-Trap-Transistor (CTT) , 2017, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[7]  Kristi Morgansen,et al.  Monocular distance estimation from optic flow during active landing maneuvers , 2014, Bioinspiration & biomimetics.

[8]  Holger G. Krapp,et al.  Wide-field, motion-sensitive neurons and matched filters for optic flow fields , 2000, Biological Cybernetics.

[9]  B. Hassenstein,et al.  Systemtheoretische Analyse der Zeit-, Reihenfolgen- und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers Chlorophanus , 1956 .

[10]  Alexander Borst,et al.  How fly neurons compute the direction of visual motion , 2019, Journal of Comparative Physiology A.

[11]  Roland Siegwart,et al.  Fusion of IMU and Vision for Absolute Scale Estimation in Monocular SLAM , 2011, J. Intell. Robotic Syst..

[12]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[13]  R. Hengstenberg,et al.  Estimation of self-motion by optic flow processing in single visual interneurons , 1996, Nature.

[14]  Daniel Cremers,et al.  Direct Sparse Odometry , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[16]  Reid R. Harrison,et al.  A biologically inspired analog IC for visual collision detection , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[17]  Guoquan Huang,et al.  Visual-Inertial Navigation: A Concise Review , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[18]  C. Desplan,et al.  The diversity of lobula plate tangential cells (LPTCs) in the Drosophila motion vision system , 2019, Journal of Comparative Physiology A.

[19]  Thomas R Clandinin,et al.  Elementary Motion Detection in Drosophila: Algorithms and Mechanisms. , 2018, Annual review of vision science.

[20]  Thomas Brox,et al.  DeMoN: Depth and Motion Network for Learning Monocular Stereo , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Daniel Cremers,et al.  Direct Sparse Visual-Inertial Odometry Using Dynamic Marginalization , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Robert J. Wood,et al.  Science, technology and the future of small autonomous drones , 2015, Nature.

[24]  Robin R. Murphy,et al.  A VLSI Architecture and Algorithm for Lucas–Kanade-Based Optical Flow Computation , 2010, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[25]  Michael J. Black,et al.  Optical Flow in Mostly Rigid Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Kevin Y. Ma,et al.  Controlled Flight of a Biologically Inspired, Insect-Scale Robot , 2013, Science.

[27]  Michael H. Dickinson,et al.  Flies Regulate Wing Motion via Active Control of a Dual-Function Gyroscope , 2019, Current Biology.

[28]  Floris van Breugel,et al.  FigureFirst: A Layout-first Approach for Scientific Figures , 2017 .

[29]  R. Hetherington The Perception of the Visual World , 1952 .

[30]  Michael H Dickinson,et al.  The visual control of landing and obstacle avoidance in the fruit fly Drosophila melanogaster , 2012, Journal of Experimental Biology.

[31]  J. Gibson The perception of the visual world , 1951 .

[32]  James Sean Humbert,et al.  Implementation of wide-field integration of optic flow for autonomous quadrotor navigation , 2009, Auton. Robots.

[33]  Paul E. Hasler,et al.  A Highly Dense, Low Power, Programmable Analog Vector-Matrix Multiplier: The FPAA Implementation , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[34]  Marie P Suver,et al.  An Array of Descending Visual Interneurons Encoding Self-Motion in Drosophila , 2016, The Journal of Neuroscience.

[35]  Martin Egelhaaf,et al.  On the neuronal basis of figure-ground discrimination by relative motion in the visual system of the fly , 1985, Biological Cybernetics.

[36]  Michael B. Reiser,et al.  Visual projection neurons in the Drosophila lobula link feature detection to distinct behavioral programs , 2016, eLife.

[37]  Michael B. Reiser,et al.  Ultra-selective looming detection from radial motion opponency , 2017, Nature.

[38]  B Schnell,et al.  Processing of horizontal optic flow in three visual interneurons of the Drosophila brain. , 2010, Journal of neurophysiology.

[39]  Michael Bosse,et al.  Keyframe-based visual–inertial odometry using nonlinear optimization , 2015, Int. J. Robotics Res..

[40]  J. Pringle The gyroscopic mechanism of the halteres of Diptera , 1948, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.

[41]  Juan D. Tardós,et al.  Visual-Inertial Monocular SLAM With Map Reuse , 2016, IEEE Robotics and Automation Letters.

[42]  Alessandro Verri,et al.  Absolute depth estimate in stereopsis , 1986 .