Kinect-based people detection and tracking from small-footprint ground robots

Small-footprint mobile ground robots, such as the popular Turtlebot and Kobuki platforms, are by necessity equipped with sensors which lie close to the ground. Reliably detecting and tracking people from this viewpoint is a challenging problem, whose solution is a key requirement for many applications involving sharing of common spaces and close human-robot interaction. We present a robust solution for cluttered indoor environments, using an inexpensive RGB-D sensor such as the Microsoft Kinect or Asus Xtion. Even in challenging scenarios with multiple people in view at once and occluding each other, our system solves the person detection problem significantly better than alternative approaches, reaching a precision, recall and F1-score of 0.85, 0.81 and 0.83, respectively. Evaluation datasets, a real-time ROS-enabled implementation and demonstration videos are provided as supplementary material.

[1]  S.S. Blackman,et al.  Multiple hypothesis tracking for multiple target tracking , 2004, IEEE Aerospace and Electronic Systems Magazine.

[2]  Luca Maria Gambardella,et al.  Perceiving People from a Low-lying Viewpoint , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Roland Siegwart,et al.  A Layered Approach to People Detection in 3D Range Data , 2010, AAAI.

[4]  Takashi Tsubouchi,et al.  People Tracking Using a Robot in Motion with Laser Range Finder , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[6]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[7]  Kai Oliver Arras,et al.  People detection in RGB-D data , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Ryo Kurazume,et al.  Multi-Part People Detection Using 2D Range Data , 2010, Int. J. Soc. Robotics.

[9]  Luc Van Gool,et al.  Random Forests for Real Time 3D Face Analysis , 2012, International Journal of Computer Vision.

[10]  Wolfram Burgard,et al.  Using Boosted Features for the Detection of People in 2D Range Data , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[11]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[12]  António E. Ruano,et al.  Fast Line, Arc/Circle and Leg Detection from Laser Scan Data in a Player Driver , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[13]  Frederik Hegger,et al.  People Detection in 3d Point Clouds Using Local Surface Normals , 2012, RoboCup.

[14]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[15]  Luca Maria Gambardella,et al.  Human-friendly robot navigation in dynamic environments , 2013, 2013 IEEE International Conference on Robotics and Automation.

[16]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[17]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[18]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Rachid Alami,et al.  Navigation in the presence of humans , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[20]  Takayuki Kanda,et al.  Abstracting People's Trajectories for Social Robots to Proactively Approach Customers , 2009, IEEE Transactions on Robotics.

[21]  Wolfram Burgard,et al.  Efficient people tracking in laser range data using a multi-hypothesis leg-tracker with adaptive occlusion probabilities , 2008, 2008 IEEE International Conference on Robotics and Automation.

[22]  Thomas Bak,et al.  Adaptive Human-Aware Robot Navigation in Close Proximity to Humans , 2011 .

[23]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[24]  Maja J. Mataric,et al.  A laser-based people tracker , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[25]  Wolfram Burgard,et al.  People Tracking with Mobile Robots Using Sample-Based Joint Probabilistic Data Association Filters , 2003, Int. J. Robotics Res..

[26]  Ba-Ngu Vo,et al.  The Gaussian Mixture Probability Hypothesis Density Filter , 2006, IEEE Transactions on Signal Processing.

[27]  Y. Bar-Shalom,et al.  The probabilistic data association filter , 2009, IEEE Control Systems.

[28]  Yasushi Nakauchi,et al.  A Social Robot that Stands in Line , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[29]  Jake K. Aggarwal,et al.  Human detection using depth information by Kinect , 2011, CVPR 2011 WORKSHOPS.

[30]  Matteo Munaro,et al.  Fast and Robust Multi-people Tracking from RGB-D Data for a Mobile Robot , 2012, IAS.

[31]  Li-Chen Fu,et al.  Human-Centered Robot Navigation—Towards a Harmoniously Human–Robot Coexisting Environment , 2011, IEEE Transactions on Robotics.