Towards Accurate 3D Person Detection and Localization from RGB-D in Cluttered Environments

We focus on the problem of accurately detecting and localizing 3D centroids of persons in RGB-D scenes with frequent heavy occlusions, as often encountered in industrial and service robotics use-cases. While recently, enormous progress has been made in 2D object detection, which is often evaluated in terms of bounding box overlap in image space, robotics systems often rely on metric 3D world coordinates for applications such as human tracking across sensor boundaries, socially aware motion planning or safety and collision avoidance. Starting with a state-of-the-art 2D single-stage detector, we examine how we can robustly lift the coordinates into 3D to outperform the state-of-the-art in RGB-D person detection at 50 frames per second. Evaluation on our Kinect v2 dataset from an intralogistics warehouse indicates that there might be better intermediate representations for this purpose than 2D bounding boxes, such as instance segmentation masks or keypoint estimates. As an alternative strategy, we also compare our method against a recently proposed bottom-up 3D human pose estimation approach. We find that our 2D top-down person detector achieves higher maximum recall, while the bottom-up 3D human pose estimation method can reach higher precision.