Real-Time Multi-Modal People Detection and Tracking of Mobile Robots with A RGB-D Sensor*

A real-time multi-modal people detection and tracking algorithm based on RGB-D information is presented in this paper. The color image and the depth image is calibrated and aligned firstly with each other. At texture level a SVM classifier based on HOG features is used to detect and identify human body in color images, while at depth level an Adaboost classifier with Haar-like features is used to detect human head in depth images respectively. With the space relationship between human body and head, a combined multi-modal cascade detector is then constructed, which can greatly improve the detection accuracy. For tracking tasks, the KCF(Kernelized Correlation Filter) algorithm is adopted to realize the real-time people tracking, in which color and depth features are both considered and used in template matching to guarantee the robustness of the tracking algorithm. The proposed algorithm is robust to object occlusion and tracking lost caused by fast movement or cross walking. Extensive experiments are carried out on a wheeled mobile robot mounted with a Kinect2 sensor, and the result shows the feasibility and effectiveness of the method.