Self-supervised monocular distance learning on a lightweight micro air vehicle

Obstacle detection by monocular vision is challenging because a single camera does not provide a direct measure for absolute distances to objects. A self-supervised learning approach is proposed that combines a camera and a very small short-range proximity sensor to find the relation between the appearance of objects in camera images and their corresponding distances. The method is efficient enough to run real time on a small camera system that can be carried onboard a lightweight MAV of 19 g. The effectiveness of the method is demonstrated by computer simulations and by experiments with the real platform in flight.

[1]  P. V. Komi,et al.  Optic fibre as a transducer of tendomuscular forces , 2004, European Journal of Applied Physiology and Occupational Physiology.

[2]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[3]  Guido C. H. E. de Croon,et al.  Autonomous flight of a 20-gram Flapping Wing MAV with a 4-gram onboard stereo vision system , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Carl E. Rasmussen,et al.  Learning Depth from Stereo , 2004, DAGM-Symposium.

[5]  Larry H. Matthies,et al.  Stereo vision-based obstacle avoidance for micro air vehicles using disparity space , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Karthik Dantu,et al.  Autonomous MAV guidance with a lightweight omnidirectional vision sensor , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[8]  H. W. Ho,et al.  Optical-flow based self-supervised learning of obstacle appearance applied to MAV landing , 2018, Robotics Auton. Syst..

[9]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[10]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[11]  Davide Scaramuzza,et al.  REMODE: Probabilistic, monocular dense reconstruction in real time , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Daniel D. Lee,et al.  Online self-supervised monocular visual odometry for ground vehicles , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Sebastian Thrun,et al.  Self-supervised Monocular Road Detection in Desert Terrain , 2006, Robotics: Science and Systems.

[14]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  MicroAir,et al.  Monocular distance estimation with optical flow maneuvers and efference copies: a stability-based strategy , 2016, Bioinspiration & Biomimetics.

[16]  G C H E de Croon,et al.  Design, aerodynamics and autonomy of the DelFly , 2012, Bioinspiration & biomimetics.

[17]  Martial Hebert,et al.  Learning monocular reactive UAV control in cluttered natural environments , 2012, 2013 IEEE International Conference on Robotics and Automation.

[18]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[19]  Dario Floreano,et al.  A 10-gram vision-based flying robot , 2007, Adv. Robotics.

[20]  Daniel Cremers,et al.  Camera-based navigation of a low-cost quadrocopter , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[22]  Daniel Cremers,et al.  Collision Avoidance for Quadrotors with a Monocular Camera , 2014, ISER.

[23]  Daniel Cremers,et al.  Visual-Inertial Navigation for a Camera-Equipped 25g Nano-Quadrotor , 2014 .

[24]  Dario Floreano,et al.  A Collision‐resilient Flying Robot , 2014, J. Field Robotics.

[25]  Vijay Kumar,et al.  Tightly-coupled monocular visual-inertial fusion for autonomous flight of rotorcraft MAVs , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[26]  K. Madhava Krishna,et al.  Autonomous navigation of generic monocular quadcopter in natural environment , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Russ Tedrake,et al.  Pushbroom stereo for high-speed navigation in cluttered environments , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Robert J. Wood,et al.  Altitude feedback control of a flapping-wing microrobot using an on-board biologically inspired optical flow sensor , 2012, 2012 IEEE International Conference on Robotics and Automation.

[29]  Dario Floreano,et al.  Optic-Flow Based Control of a 46g Quadrotor , 2013 .

[30]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[31]  Fabrizio Angiulli,et al.  Fast condensed nearest neighbor rule , 2005, ICML.

[32]  Yun-Su Ha,et al.  Environmental map building for a mobile robot using infrared range-finder sensors , 2004, Adv. Robotics.

[33]  S. Thrun,et al.  Optical Flow Approaches for Self-supervised Learning in Autonomous Mobile Robot Navigation , 2007 .

[34]  Wolfram Burgard,et al.  Monocular range sensing: A non-parametric learning approach , 2008, 2008 IEEE International Conference on Robotics and Automation.

[35]  Marcin Blachnik,et al.  Fusion of instance selection methods in regression tasks , 2016, Inf. Fusion.