Estimation of Kinect depth confidence through self-training

All depth data captured by Kinect devices are noisy, and sometimes even lost or shifted, especially around the edges of the depth. In this paper, we propose an approach to generate a per-pixel confidence measurement for each depth map captured by Kinect devices in indoor environments through supervised learning. Several distinguishing features from both the color images and depth maps are selected to train depth map estimators using Random Forest regressor. Using this estimator, we can predict a confidence map of any depth map captured by Kinect devices. Usage of other devices, such as an industrial laser scanner, is unnecessary, making the implementation more convenient. The experiments demonstrate precise confidence prediction of the depth.

[1]  Yo-Sung Ho,et al.  Discontinuity-adaptive depth map filtering for 3D view generation , 2009, IMMERSCOM.

[2]  Mongi A. Abidi,et al.  Spatial and Temporal Enhancement of Depth Images Captured by a Time-of-Flight Depth Sensor , 2010, 2010 20th International Conference on Pattern Recognition.

[3]  Matti Pietikäinen,et al.  Performance evaluation of texture measures with classification based on Kullback discrimination of distributions , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[4]  Wa James Tam,et al.  Nonuniform smoothing of depth maps before image-based rendering , 2004, SPIE Optics East.

[5]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[6]  Xi Yang,et al.  Fusion of depth maps based on confidence , 2011, 2011 International Conference on Electronics, Communications and Control (ICECC).

[7]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[8]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision , 2013, Advances in Computer Vision and Pattern Recognition.

[9]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[10]  Juho Kannala,et al.  Joint Depth and Color Camera Calibration with Distortion Correction , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Roland Siegwart,et al.  Extrinsic self calibration of a camera and a 3D laser range finder from natural scenes , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Tim Weyrich,et al.  Capturing Time-of-Flight data with confidence , 2011, CVPR 2011.

[13]  Andreas Nüchter,et al.  Robust 3D-mapping with time-of-flight cameras , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Robert Pless,et al.  Extrinsic calibration of a camera and laser range finder (improves camera calibration) , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[15]  D. Holz,et al.  3 D Pose Estimation and Mapping with Time-of-Flight Cameras , 2008 .

[16]  J. Daugman Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[17]  Liang-Gee Chen,et al.  Efficient Depth Image Based Rendering with Edge Dependent Depth Filter and Interpolation , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[18]  Juho Kannala,et al.  Accurate and Practical Calibration of a Depth and Color Camera Pair , 2011, CAIP.

[19]  Shahram Izadi,et al.  Modeling Kinect Sensor Noise for Improved 3D Reconstruction and Tracking , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[20]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.