Real-time people tracking for mobile robots using thermal vision

Abstract This paper presents a vision-based approach for tracking people on a mobile robot using thermal images. The approach combines a particle filter with two alternative measurement models that are suitable for real-time tracking. With this approach a person can be detected independently from current light conditions and in situations where no skin colour is visible. In addition, the paper presents a comprehensive, quantitative evaluation of the different methods on a mobile robot in an office environment, for both single and multiple persons. The results show that the measurement model that was learned from local grey-scale features could improve on the performance of the elliptic contour model, and that both models could be combined to further improve performance with minimal extra computational cost.

[1]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[2]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[3]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Lawrence D. Stone,et al.  Bayesian Multiple Target Tracking , 1999 .

[5]  Horst-Michael Groß,et al.  A multi-modal system for tracking and analyzing faces on a mobile robot , 2004, Robotics Auton. Syst..

[6]  James E. Black,et al.  A novel method for video tracking performance evaluation , 2003 .

[7]  Peihua Li,et al.  Probabilistic Object Tracking Based on Machine Learning and Importance Sampling , 2005, IbPRIA.

[8]  Tieniu Tan,et al.  People tracking based on motion model and motion constraints with automatic initialization , 2004, Pattern Recognit..

[9]  Roger D. Boyle,et al.  Performance Evaluation Metrics and Statistics for Positional Tracker Evaluation , 2003, ICVS.

[10]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[11]  Maja J. Mataric,et al.  A laser-based people tracker , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[12]  Universityof SouthernCalifornia LosAngeles.  Laser-based People Tracking , 2002 .

[13]  David S. Doermann,et al.  Tools and techniques for video performance evaluation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[14]  Andreas Zell,et al.  Detection, tracking, and pursuit of humans with an autonomous mobile robot , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[15]  Paulo Menezes,et al.  Human-robot interaction based on Haar-like features and eigenfaces , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[16]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[17]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[18]  Daniel Gatica-Perez,et al.  Order Matters: A Distributed Sampling Method for Multi-Object Tracking , 2004, BMVC.

[19]  William Whittaker,et al.  Conditional particle filters for simultaneous mobile robot localization and people-tracking , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[20]  Larry S. Davis,et al.  Probabilistic template based pedestrian detection in infrared videos , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[21]  A. Murat Tekalp,et al.  Metrics for performance evaluation of video object segmentation and tracking without ground-truth , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[22]  Ben J. A. Kröse,et al.  Keeping Track of Humans: Have I Seen This Person Before? , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[23]  Wolfram Burgard,et al.  Tracking multiple moving targets with a mobile robot using particle filters and statistical data association , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[24]  Grzegorz Cielniak,et al.  Person identification by mobile robots in indoor environments , 2003, 1st International Workshop on Robotic Sensing, 2003. ROSE' 03..

[25]  Christian Schlegel,et al.  Vision Based Person Tracking with a Mobile Robot , 1998, BMVC.

[26]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[27]  Paulo Menezes,et al.  Face tracking and hand gesture recognition for human-robot interaction , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[28]  Michael Isard,et al.  BraMBLe: a Bayesian multiple-blob tracker , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[29]  Heiko Neumann,et al.  Detection and classification of obstacles in night vision traffic scenes based on infrared imagery , 2003, Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems.

[30]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[31]  Wolfram Burgard,et al.  Fast face detection for mobile robots by integrating laser range data with vision , 2003 .

[32]  Paulo Menezes,et al.  Visual Tracking of Silhouettes for Human-Robot Interaction , 2003 .

[33]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[34]  Wolfram Burgard,et al.  People Tracking with Mobile Robots Using Sample-Based Joint Probabilistic Data Association Filters , 2003, Int. J. Robotics Res..

[35]  Frank Dellaert,et al.  An MCMC-Based Particle Filter for Tracking Multiple Interacting Targets , 2004, ECCV.

[36]  Alberto Broggi,et al.  Pedestrian detection in infrared images , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[37]  Xia Liu,et al.  Pedestrian detection and tracking with night vision , 2005, IEEE Transactions on Intelligent Transportation Systems.

[38]  Sebastian Lang,et al.  Providing the basis for human-robot-interaction: a multi-modal attention system for a mobile robot , 2003, ICMI '03.