Efficient Optical Flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone

Micro Aerial Vehicles (FOV) are very suitable for flying in indoor environments, but autonomous navigation is challenging due to their strict hardware limitations. This paper presents a highly efficient computer vision algorithm called Edge-FS for the determination of velocity and depth. It runs at 20 Hz on a 4 g stereo camera with an embedded STM32F4 microprocessor (168 MHz, 192 kB) and uses edge distributions to calculate optical flow and stereo disparity. The stereo-based distance estimates are used to scale the optical flow in order to retrieve the drone's velocity. The velocity and depth measurements are used for fully autonomous flight of a 40 g pocket drone only relying on on-board sensors. This method allows the MAV to control its velocity and avoid obstacles.

[1]  Paul Y. Oh,et al.  Optic-Flow-Based Collision Avoidance , 2008, IEEE Robotics & Automation Magazine.

[2]  Max Mulder,et al.  Obstacle Avoidance Strategy using Onboard Stereo Vision on a Flapping Wing MAV , 2016, IEEE Transactions on Robotics.

[3]  Rogelio Lozano,et al.  Real-Time Stabilization of an Eight-Rotor UAV Using Optical Flow , 2009, IEEE Transactions on Robotics.

[4]  Marc Pollefeys,et al.  Reactive avoidance using embedded stereo vision for MAV flight , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[5]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[6]  Kenzo Nonami,et al.  Optic flow-based vision system for autonomous 3D localization and control of small aerial vehicles , 2009, Robotics Auton. Syst..

[7]  Guido C. H. E. de Croon,et al.  Adaptive Incremental Nonlinear Dynamic Inversion for Attitude Control of Micro Air Vehicles , 2016 .

[8]  Randal W. Beard,et al.  See and avoidance behaviors for autonomous navigation , 2004, SPIE Optics East.

[9]  Hilbert J. Kappen,et al.  Local histogram matching for efficient optical flow computation applied to velocity estimation on pocket drones , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Sebastian Scherer,et al.  First results in detecting and avoiding frontal obstacles from a monocular camera for micro unmanned aerial vehicles , 2013, 2013 IEEE International Conference on Robotics and Automation.

[11]  Dario Floreano,et al.  A method for ego-motion estimation in micro-hovering platforms flying in very cluttered environments , 2016, Auton. Robots.

[12]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[13]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[14]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Guido C. H. E. de Croon,et al.  Autonomous flight of a 20-gram Flapping Wing MAV with a 4-gram onboard stereo vision system , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Karthik Dantu,et al.  Autonomous MAV guidance with a lightweight omnidirectional vision sensor , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Dario Floreano,et al.  Optic-Flow Based Control of a 46g Quadrotor , 2013 .

[19]  Daniel Cremers,et al.  Visual-Inertial Navigation for a Camera-Equipped 25g Nano-Quadrotor , 2014 .

[20]  Marc Pollefeys,et al.  An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications , 2013, 2013 IEEE International Conference on Robotics and Automation.

[21]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[22]  Xiaoyan Hu,et al.  Evaluation of stereo confidence indoors and outdoors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Heinrich H. Bülthoff,et al.  Nonlinear ego-motion estimation from optical flow for online control of a quadrotor UAV , 2015, Int. J. Robotics Res..

[24]  Stéphane Viollet,et al.  Bio-inspired optical flow circuits for the visual guidance of micro air vehicles , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[25]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[26]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.