Color Guided Depth Map Super-Resolution based on a Deep Self-Learning Approach

RGB-D cameras have been widely used in various applications, such as gesturing or exercise games, consumer healthcare systems, and 3D measurements. However, the resolution of depth maps is lower than that of an RGB image, which significantly limits the potential applications of depth maps. Recently, deep learning-based super-resolution techniques have achieved state-of-the-art results for image resolution enhancement, in which paired high- and low-resolution images are used for training. The challenge with the depth map super resolution for RGB-D cameras is that we do not have high-resolution depth maps for training models (deep networks). We propose a deep self-learning approach for color-guided depth map super resolution. We achieve super resolution using only low-resolution depth maps to train a network, which is comparable to the ideal case (training the network using paired high- and low-resolution images. The high-resolution image is used as a guide to improve the resolution enhancement. Experimental results demonstrate that the proposed method outperforms almost as same as accuracy of trained by external datasets even not using external datasets.