A computationally efficient denoising and hole-filling method for depth image enhancement

Depth maps captured by Kinect depth cameras are being widely used for 3D action recognition. However, such images often appear noisy and contain missing pixels or black holes. This paper presents a computationally efficient method for both denoising and hole-filling in depth images. The denoising is achieved by utilizing a combination of Gaussian kernel filtering and anisotropic filtering. The hole-filling is achieved by utilizing a combination of morphological filtering and zero block filtering. Experimental results using the publicly available datasets are provided indicating the superiority of the developed method in terms of both depth error and computational efficiency compared to three existing methods.

[1]  Nasser Kehtarnavaz,et al.  Home-based Senior Fitness Test measurement system using collaborative inertial and depth sensors , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[2]  Ying Wu,et al.  Discriminative Spatial Attention for Robust Tracking , 2010, ECCV.

[3]  Nasser Kehtarnavaz,et al.  Improving Human Action Recognition Using Fusion of Depth Camera and Inertial Sensors , 2015, IEEE Transactions on Human-Machine Systems.

[4]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Chongyu Chen,et al.  A color-guided, region-adaptive and depth-selective unified framework for Kinect depth recovery , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[6]  Nasser Kehtarnavaz,et al.  A medication adherence monitoring system for pill bottles based on a wearable inertial sensor , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Xiaojin Gong,et al.  Guided inpainting and filtering for Kinect depth maps , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[8]  Thomas B. Moeslund,et al.  A selective spatio-temporal interest point detector for human action recognition in complex scenes , 2011, 2011 International Conference on Computer Vision.

[9]  Bingbing Ni,et al.  Motion Part Regularization: Improving action recognition via trajectory group selection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[11]  Seung-Won Jung,et al.  Directional Joint Bilateral Filter for Depth Images , 2014, Sensors.

[12]  Yun Yang,et al.  Gradient Local Auto-Correlations and Extreme Learning Machine for Depth-Based Activity Recognition , 2015, ISVC.

[13]  Seung-Won Jung,et al.  Enhancement of Image and Depth Map Using Adaptive Joint Trilateral Filter , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Emile A. Hendriks,et al.  Hybrid Kinect Depth Map Refinement for Transparent Objects , 2014, 2014 22nd International Conference on Pattern Recognition.

[15]  Haibo Wang,et al.  Kinect depth restoration via energy minimization with TV21 regularization , 2013, 2013 IEEE International Conference on Image Processing.

[16]  Nasser Kehtarnavaz,et al.  Action Recognition from Depth Sequences Using Depth Motion Maps-Based Local Binary Patterns , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[17]  Luis Salgado,et al.  Efficient spatio-temporal hole filling strategy for Kinect depth maps , 2012, Electronic Imaging.

[18]  Nicolai Petkov,et al.  Contour and boundary detection improved by surround suppression of texture edges , 2004, Image Vis. Comput..

[19]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[20]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[21]  Bogdan Kwolek,et al.  Human fall detection on embedded platform using depth maps and wireless accelerometer , 2014, Comput. Methods Programs Biomed..

[22]  Yao Wang,et al.  Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model , 2014, IEEE Transactions on Image Processing.

[23]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[24]  Jake K. Aggarwal,et al.  View invariant human action recognition using histograms of 3D joints , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[25]  Nasser Kehtarnavaz,et al.  A Real-Time Human Action Recognition System Using Depth and Inertial Sensor Fusion , 2016, IEEE Sensors Journal.

[26]  Elsayed E. Hemayed,et al.  Human action recognition using trajectory-based representation , 2015 .