RGB-D Object Tracking with Occlusion Detection

Tracking objects in RGB-D images is a challenging task in computer vision, especially under occlusion. In this paper, we proposed an object tracking method based on the 3D point cloud. Firstly, we convert RGB-D images to point clouds. Secondly, features of point clouds are extracted by PointNet and finally integrated into the 3D object tracking algorithm for template matching across frames. A strategy of occlusion detection and target retrieval is applied to handle target missing under occlusion. For example, when the number of point clouds is decreasing abruptly, the occlusion may take place. Then a YOLOv3 detector is used to re-find this target. Our network is insensitive to appearance variation of object and robust to object tracking. The experimental results show that the proposed method achieves comparable results to state of the art on the Princeton RGB-D Tracking Benchmark.

[1]  Jiri Matas,et al.  How to Make an RGBD Tracker? , 2018, ECCV Workshops.

[2]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[3]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[5]  Jiri Matas,et al.  Object Tracking by Reconstruction With View-Specific Discriminative Correlation Filters , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Majid Mirmehdi,et al.  Real-time RGB-D Tracking with Depth Scaling Kernelised Correlation Filters and Occlusion Handling , 2015, BMVC.

[7]  Jianxiong Xiao,et al.  Tracking Revisited Using RGBD Camera: Unified Benchmark and Baselines , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[9]  Jianke Zhu,et al.  A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration , 2014, ECCV Workshops.

[10]  Jiri Matas,et al.  Depth Masked Discriminative Correlation Filter , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[11]  Xiao-Yuan Jing,et al.  Context-Aware Three-Dimensional Mean-Shift With Occlusion Handling for Robust Object Tracking in RGB-D Videos , 2019, IEEE Transactions on Multimedia.

[12]  Shin Ishii,et al.  An occlusion-aware particle filter tracker to handle complex and persistent occlusions , 2016, Computer Vision and Image Understanding.

[13]  Massimo Piccardi,et al.  Prototype-based budget maintenance for tracking in depth videos , 2016, Multimedia Tools and Applications.

[14]  Tianzhu Zhang,et al.  3D Part-Based Sparse Tracker with Automatic Synchronization and Registration , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).