Monocular Localization of a moving person onboard a Quadrotor MAV

In this paper, we propose a novel method to recover the 3D trajectory of a moving person from a monocular camera mounted on a quadrotor micro aerial vehicle (MAV). The key contribution is an integrated approach that simultaneously performs visual odometry (VO) and persistent tracking of a person automatically detected in the scene. All computation pertaining to VO, detection and tracking runs onboard the MAV from a front-facing monocular RGB camera. Given the gravity direction from an inertial sensor and the knowledge of the individual's height, a complete 3D trajectory of the person within the reconstructed scene can be estimated. When the ground plane is detected from the triangulated 3D points, the absolute metric scale of the trajectory and the 3D map is also recovered. Our extensive indoor and outdoor experiments show that the system can localize a person moving naturally within a large area. The system runs at 17 frames per second on the onboard computer. A walking person was successfully tracked for two minutes and an accurate trajectory was recovered over a distance of 140 meters with our system running onboard.

[1]  Roland Siegwart,et al.  A Toolbox for Easily Calibrating Omnidirectional Cameras , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Horst Bischof,et al.  Dense reconstruction on-the-fly , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Jose-Ernesto Gomez-Balderas,et al.  Tracking a Ground Moving Target with a Quadrotor Using Switching Control , 2012, Journal of Intelligent & Robotic Systems.

[7]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  Michael F. Cohen,et al.  Real-time image-based 6-DOF localization in large-scale environments , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Chieh-Chih Wang,et al.  Stereo-based simultaneous localization, mapping and moving object tracking , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Éric Marchand,et al.  Chasing a moving target from a flying UAV , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Pascual Campoy Cervera,et al.  Computer vision based general object following for GPS-denied multirotor unmanned vehicles , 2014, 2014 American Control Conference.

[12]  Daniel Cremers,et al.  FollowMe: Person following and gesture recognition with a quadrocopter , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Pietro Perona,et al.  Integral Channel Features , 2009, BMVC.

[14]  Vibhav Vineet,et al.  Struck: Structured Output Tracking with Kernels , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Kevin D. Jones,et al.  Vision-based tracking and motion estimation for moving targets using small UAVs , 2006, 2006 American Control Conference.

[16]  Vijay Kumar,et al.  Vision-based state estimation for autonomous rotorcraft MAVs in complex environments , 2013, 2013 IEEE International Conference on Robotics and Automation.

[17]  Trung-Dung Vu,et al.  Online Localization and Mapping with Moving Object Tracking in Dynamic Outdoor Environments , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[18]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[19]  Yi Wu,et al.  Online Object Tracking: A Benchmark , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Hugh F. Durrant-Whyte,et al.  Simultaneous Localization, Mapping and Moving Object Tracking , 2007, Int. J. Robotics Res..

[21]  K. Madhava Krishna,et al.  Realtime multibody visual SLAM with a smoothly moving monocular camera , 2011, 2011 International Conference on Computer Vision.

[22]  Daniel Cremers,et al.  Semi-dense Visual Odometry for a Monocular Camera , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Roland Siegwart,et al.  Onboard IMU and monocular vision based control for MAVs in unknown in- and outdoor environments , 2011, 2011 IEEE International Conference on Robotics and Automation.

[24]  Isaac Kaminer,et al.  Vision-Based Tracking and Motion Estimation for Moving Targets Using Small UAVs , 2006 .

[25]  H. Jin Kim,et al.  Real-time 6-DOF monocular visual SLAM in a large-scale environment , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Marc Pollefeys,et al.  PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision , 2012, Auton. Robots.

[27]  Jiri Matas,et al.  A system for real-time detection and tracking of vehicles from a single car-mounted camera , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[28]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[29]  Daniel Cremers,et al.  Accurate Figure Flying with a Quadrocopter Using Onboard Visual and Inertial Sensing , 2012 .

[30]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[31]  Jonathan Brandt,et al.  Robust object detection via soft cascade , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Marc Pollefeys,et al.  Autonomous Visual Mapping and Exploration With a Micro Aerial Vehicle , 2014, J. Field Robotics.

[33]  Stergios I. Roumeliotis,et al.  A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[34]  Roland Siegwart,et al.  Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments , 2012, 2012 IEEE International Conference on Robotics and Automation.

[35]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.