People Detection, Tracking and Visualization Using ROS on a Mobile Service Robot

In this case study chapter, we discuss the implementation and deployment of a ROS-based, multi-modal people detection and tracking framework on a custom-built mobile service robot during the EU FP7 project SPENCER. The mildly humanized robot platform is equipped with five computers and an array of RGB-D, stereo and 2D laser range sensors. After describing the robot platform, we illustrate our real-time perception pipeline starting from ROS-based people detection modules for RGB-D and 2D laser data, via nodes for aggregating detections from multiple sensors, up to person and group tracking. For each stage of the pipeline, we provide sample code online. We also present a set of highly configurable, custom RViz plugins for visualizing detected and tracked persons and groups. Due to the flexible and modular structure of our pipeline, all of our components can easily be reused in custom setups. Finally, we outline how to generate test data using a pedestrian simulator and Gazebo. We conclude with quantitative results from our experiments and lessons that we learned during the project. To our knowledge, the presented framework is the functionally most complete one that is currently available for ROS as open-source software.

[1]  Kai Oliver Arras,et al.  Better models for people tracking , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Wolfram Burgard,et al.  Using Boosted Features for the Detection of People in 2D Range Data , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[3]  Bastian Leibe,et al.  Real-time RGB-D based people detection and tracking for mobile robots and head-worn cameras , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Ba-Ngu Vo,et al.  A Consistent Metric for Performance Evaluation of Multi-Object Filters , 2008, IEEE Transactions on Signal Processing.

[5]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[6]  Kai Oliver Arras,et al.  People detection in RGB-D data , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Luc Van Gool,et al.  Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings , 2010, ECCV.

[8]  Kai Oliver Arras,et al.  Multi-model hypothesis tracking of groups of people in RGB-D data , 2014, 17th International Conference on Information Fusion (FUSION).

[9]  Kai Oliver Arras,et al.  Multi-Hypothesis Social Grouping and Tracking for Mobile Robots , 2013, Robotics: Science and Systems.

[10]  D. Helbing,et al.  The Walking Behaviour of Pedestrian Social Groups and Its Impact on Crowd Dynamics , 2010, PloS one.

[11]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[12]  Wolfram Burgard,et al.  Multi-model Hypothesis Group Tracking and Group Size Estimation , 2010, Int. J. Soc. Robotics.

[13]  Samuel S. Blackman,et al.  Design and Analysis of Modern Tracking Systems , 1999 .

[14]  Zhen Qin,et al.  Improving multi-target tracking via social grouping , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Matteo Munaro,et al.  Tracking people within groups with RGB-D data , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Ting Yu,et al.  Monitoring, recognizing and discovering social networks , 2009, CVPR.

[17]  Wolfram Burgard,et al.  People Tracking with Mobile Robots Using Sample-Based Joint Probabilistic Data Association Filters , 2003, Int. J. Robotics Res..

[18]  Alper Yilmaz,et al.  Inferring social relations from visual concepts , 2011, 2011 International Conference on Computer Vision.

[19]  Ajo Fod,et al.  Laser-Based People Tracking , 2002 .

[20]  Kai Oliver Arras,et al.  People tracking in RGB-D data with on-line boosted target models , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Gang Wang,et al.  Seeing People in Social Context: Recognizing People and Social Relationships , 2010, ECCV.

[22]  Kai Oliver Arras,et al.  People tracking with human motion predictions from social forces , 2010, 2010 IEEE International Conference on Robotics and Automation.

[23]  Wolfram Burgard,et al.  Efficient people tracking in laser range data using a multi-hypothesis leg-tracker with adaptive occlusion probabilities , 2008, 2008 IEEE International Conference on Robotics and Automation.

[24]  Silvio Savarese,et al.  A Unified Framework for Multi-target Tracking and Collective Activity Recognition , 2012, ECCV.

[25]  Bodo Rosenhahn,et al.  Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[26]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[27]  Ingemar J. Cox,et al.  An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking , 1996, IEEE Trans. Pattern Anal. Mach. Intell..