People perception from RGB-D cameras for mobile robots

Understanding how humans move through the scene is a key issue of decision-making for an autonomous mobile robot in crown people zones. So accurately detecting and tracking people from a mobile platform can help improve interaction effective and efficient. In this paper, we proposed a people detection and tracking system using combination of a several new techniques for mobile robots, plan-view maps, depth weighted histograms, and GNN data association. We proposed a spatial region of interest based plan-view maps to detect human candidates. Firstly, point cloud sub-clusters were segmented for candidate detection. Two different plan-view maps, named occupancy map and height map, were employed to identify human candidates from point cloud sub-clusters. Meanwhile, a depth weighted histogram was extracted to feature a human candidate. Then, a particle filter algorithm was adopted to track human's motion. Finally, data association was set up to re-identify humans which were tracked. Extensive experiments demonstrated the effectiveness and robustness of our human detection and tracking system.

[1]  Roland Siegwart,et al.  A Layered Approach to People Detection in 3D Range Data , 2010, AAAI.

[2]  Songmin Jia,et al.  Human tracking system based on adaptive multi-feature mean-shift for robot under the double-layer locating mechanism , 2014, Adv. Robotics.

[3]  Andrew Zisserman,et al.  Progressive search space reduction for human pose estimation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Silvio Savarese,et al.  A General Framework for Tracking Multiple People from a Moving Camera , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Bernt Schiele,et al.  Multi-cue onboard pedestrian detection , 2009, CVPR.

[6]  Frédéric Lerasle,et al.  Cooperative passers-by tracking with a mobile robot and external cameras , 2013, Comput. Vis. Image Underst..

[7]  Lynne E. Parker,et al.  Real-Time Multiple Human Perception With Color-Depth Cameras on a Mobile Robot , 2013, IEEE Transactions on Cybernetics.

[8]  Abdesselam Bouzerdoum,et al.  Object segmentation and classification using 3-D range camera , 2014, J. Vis. Commun. Image Represent..

[9]  Yasue Mitsukura,et al.  A human tracking mobile-robot with face detection , 2009, 2009 35th Annual Conference of IEEE Industrial Electronics.

[10]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[11]  Takayuki Kanda,et al.  Person Tracking in Large Public Spaces Using 3-D Range Sensors , 2013, IEEE Transactions on Human-Machine Systems.

[12]  Dariu Gavrila,et al.  Ieee Transactions on Intelligent Transportation Systems the Benefits of Dense Stereo for Pedestrian Detection , 2022 .

[13]  Marcelo H. Ang,et al.  A Survey on Perception Methods for Human–Robot Interaction in Social Robots , 2013, International Journal of Social Robotics.

[14]  Jwu-Sheng Hu,et al.  Design of Sensing System and Anticipative Behavior for Human Following of Mobile Robots , 2014, IEEE Transactions on Industrial Electronics.

[15]  Matteo Munaro,et al.  Fast RGB-D people tracking for service robots , 2014, Auton. Robots.

[16]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[17]  Francisco José Madrid-Cuevas,et al.  People detection and tracking with multiple stereo cameras using particle filters , 2009, J. Vis. Commun. Image Represent..

[18]  Hiroshi Ishiguro,et al.  Automatic calibration of laser range finder positions for pedestrian tracking based on social group detections , 2014, Adv. Robotics.

[19]  Ye Liu,et al.  Detecting and tracking people in real time with RGB-D camera , 2015, Pattern Recognit. Lett..

[20]  Luigi di Stefano,et al.  People Tracking Using a Time-of-Flight Depth Sensor , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[21]  Woojin Chung,et al.  The Detection and Following of Human Legs Through Inductive Approaches for a Mobile Robot With a Single Laser Range Finder , 2012, IEEE Transactions on Industrial Electronics.

[22]  Huosheng Hu,et al.  Multisensor-Based Human Detection and Tracking for Mobile Service Robots , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Aladdin Ayesh,et al.  Multi-agent system for people detection and tracking using stereo vision in mobile robots , 2009, Robotica.

[24]  Luca Maria Gambardella,et al.  Kinect-based people detection and tracking from small-footprint ground robots , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Shane Brennan,et al.  A Fast Stereo-based System for Detecting and Tracking Pedestrians from a Moving Vehicle , 2009, Int. J. Robotics Res..

[27]  ChoiWongun,et al.  A General Framework for Tracking Multiple People from a Moving Camera , 2013 .

[28]  Basilio Sierra,et al.  RGB-D, Laser and Thermal Sensor Fusion for People following in a Mobile Robot , 2013 .

[29]  Csaba Benedek,et al.  3D people surveillance on range data sequences of a rotating Lidar , 2014, Pattern Recognit. Lett..

[30]  Ying Wu,et al.  Nonstationary color tracking for vision-based human-computer interaction , 2002, IEEE Trans. Neural Networks.