Real-time Motion Tracking from a Mobile Robot

A mobile robot needs to perceive the motions of external objects to perform tasks successfully in a dynamic environment. We propose a set of algorithms for multiple motion tracking from a mobile robot equipped with a monocular camera and a laser rangefinder. The key challenges are 1. to compensate the ego-motion of the robot for external motion detection, and 2. to cope with transient and structural noise for robust motion tracking. In our algorithms, the robot ego-motion is directly estimated using corresponding feature sets in two consecutive images, and the position and velocity of a moving object is estimated in image space using multiple particle filters. The estimates are fused with the depth information from the laser rangefinder to estimate the partial 3D position. The proposed algorithms have been tested with various configurations in outdoor environments. The algorithms were deployed on three different platforms; it was shown that various type of ego-motion were successfully eliminated and the particle filters were able to track motions robustly. The real-time capability of the tracking algorithm was demonstrated by integrating it into a robot control loop.

[1]  Gaurav S. Sukhatme,et al.  Online simultaneous localization and mapping in dynamic environments , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[2]  D HagerGregory,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001 .

[3]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Gian Luca Foresti,et al.  Active Video-Based Surveillance System , 2005 .

[5]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[6]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[7]  Alonzo Kelly,et al.  A 3D State Space Formulation of a Navigation Kalman Filter for Autonomous Vehicles , 1994 .

[8]  Rama Chellappa,et al.  Image stabilization and mosaicking using the overlapped basis optical flow field , 1997, Proceedings of International Conference on Image Processing.

[9]  Evangelos E. Milios,et al.  Robot Pose Estimation in Unknown Environments by Matching 2D Range Scans , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[11]  Iwan Ulrich,et al.  VFH+: reliable obstacle avoidance for fast mobile robots , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[12]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[13]  Wolfram Burgard,et al.  A Probabilistic Approach to Concurrent Mapping and Localization for Mobile Robots , 1998, Auton. Robots.

[14]  Tim D. Barfoot,et al.  Online visual motion estimation using FastSLAM with SIFT features , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Andrew Blake,et al.  A Probabilistic Exclusion Principle for Tracking Multiple Objects , 2004, International Journal of Computer Vision.

[16]  Gregory D. Hager,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  C Tomasi,et al.  Shape and motion from image streams: a factorization method. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[19]  William Whittaker,et al.  Conditional particle filters for simultaneous mobile robot localization and people-tracking , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[20]  J.-S. Gutmann,et al.  AMOS: comparison of scan matching approaches for self-localization in indoor environments , 1996, Proceedings of the First Euromicro Workshop on Advanced Mobile Robots (EUROBOT '96).

[21]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[22]  Dieter Fox,et al.  KLD-Sampling: Adaptive Particle Filters , 2001, NIPS.

[23]  Roland Siegwart,et al.  A Trained System for Multimodal Perception in Urban Environments , 2009 .

[24]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[25]  Anup Basu,et al.  Motion Tracking with an Active Camera , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[27]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[28]  Tomas Uhlin,et al.  Closing the loop: detection and pursuit of a moving object by a moving observer , 1996, Image Vis. Comput..

[29]  Wolfram Burgard,et al.  Multi-model Hypothesis Group Tracking and Group Size Estimation , 2010, Int. J. Soc. Robotics.

[30]  Wolfram Burgard,et al.  Tracking multiple moving targets with a mobile robot using particle filters and statistical data association , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[31]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[32]  Isaac Cohen,et al.  Continuous multi-views tracking using tensor voting , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[33]  Rachid Deriche,et al.  Using geometric corners to build a 2D mosaic from a set of images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Frans C. A. Groen,et al.  Motion interpretation for in-car vision systems , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Wolfram Burgard,et al.  Robust Monte Carlo localization for mobile robots , 2001, Artif. Intell..

[36]  Josephine Sullivan,et al.  Multi-target Tracking on a Large Scale: Experiences from Football Player Tracking , 2009 .

[37]  Ingemar J. Cox,et al.  An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Gaurav S. Sukhatme,et al.  Visually guided landing of an unmanned aerial vehicle , 2003, IEEE Trans. Robotics Autom..

[39]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Rómer Rosales,et al.  3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[41]  Kai O. Arras,et al.  Spatially Grounded Multi-Hypothesis Tracking of People , 2009 .

[42]  Ali Shahrokni,et al.  A Robust Vision-based Moving Target Detection and Tracking System , 2001 .

[43]  Michal Irani,et al.  Recovery of ego-motion using image stabilization , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Gaurav S. Sukhatme,et al.  Detecting Moving Objects using a Single Camera on a Mobile Robot in an Outdoor Environment , 2004 .

[45]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[46]  Timothy D. Barfoot,et al.  VISUAL MOTION ESTIMATION AND TERRAIN MODELING FOR PLANETARY ROVERS , 2005 .

[47]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Gaurav S. Sukhatme,et al.  Cooperative target tracking using mobile robots , 2005 .

[49]  René Vidal.  Multi-Subspace Methods for Motion Segmentation from Affine, Perspective and Central Panoramic Cameras , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[50]  Patrick Pérez,et al.  Maintaining multimodality through mixture tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[51]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[52]  A. Howard,et al.  Results from a Real-time Stereo-based Pedestrian Detection System on a Moving Vehicle , 2009 .

[53]  Christian Micheloni,et al.  A robust feature tracker for active surveillance of outdoor scenes , 2003, ELCVIA Electronic Letters on Computer Vision and Image Analysis.

[54]  Greg Welch,et al.  An Introduction to Kalman Filter , 1995, SIGGRAPH 2001.

[55]  Mubarak Shah,et al.  Accurate motion layer segmentation and matting , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[56]  Wolfram Burgard,et al.  Classifying dynamic objects , 2009, Auton. Robots.

[57]  C. Hue,et al.  A particle filter to track multiple objects , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[58]  Andrea Fusiello,et al.  Image stabilization by features tracking , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[59]  A. Yilmaz,et al.  TARGET-TRACKING IN FLIR IMAGERY USING MEAN-SHIFT AND GLOBAL MOTION COMPENSATION , 2001 .

[60]  Bernt Schiele,et al.  Visual People Detection - Different Models, Comparison and Discussion , 2009, ICRA 2009.

[61]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[62]  Gaurav S. Sukhatme,et al.  A tale of two helicopters , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[63]  Andrew Blake,et al.  Statistical mosaics for tracking , 1996, Image Vis. Comput..

[64]  Konrad Schindler,et al.  Improved Multi-Person Tracking with Active Occlusion Handling , 2009, ICRA 2009.

[65]  Gérard G. Medioni,et al.  Continuous tracking within and across camera streams , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[66]  Stergios I. Roumeliotis,et al.  3-D Localization for a Mars Rover Prototype , 1999 .

[67]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[68]  Wolfram Burgard,et al.  Classifying Dynamic Objects: An Unsupervised Learning Approach , 2008, Robotics: Science and Systems.

[69]  Gérard G. Medioni,et al.  Detecting and tracking moving objects for video surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).