Objective Evaluation of Image Decomposition Algorithms for Depth Map Upsampling

Depth map upsampling plays an essential role in various three-dimensional (3D) image and video applications such as multi-view rendering and 3D scene modeling. Most of existing depth map upsampling methods have suggested to use a color image as a guide. Recently, in our previous work, the use of a structure component obtained from image decomposition rather than the color image has proven to be very powerful in the task of depth map upsampling. In this paper, to determine how image decomposition algorithms can be best used for depth map upsampling, we conducted a comprehensive comparative study. More precisely, the purpose of this study is to present an “objective” evaluation of recent promising image decomposition methods in terms of the performance of depth map upsampling. This is, to the best of our knowledge, the first experimental comparative demonstration on the performance of depth map upsampling enhanced with several different image decomposition models. We investigated eight different promising recent image decomposition approaches under the same experimental setup. From our quantitative comparison, we can obtain novel and valuable insights into the image decomposition-based depth map upsampling: (1) the best image decomposition solution for depth map upsampling depends on the scaling factor of upsampling, (2) the guided filter-based image decomposition method gives rise to the best performance for lower scaling factors, whereas the tree filter-based image decomposition method leads to the best upsampling performance for higher scaling factors, and (3) the performance of image decomposition-based depth upsampling is not sensitive to image features. We believe that this comprehensive comparative study serves as a reference point and guide for developers and practitioners in choosing an appropriate image decomposition technique adopted for building depth map upsampling systems.