Salient Error Detection based Refinement for Wide-baseline Image Interpolation

Wide-baseline image interpolation is useful in many multimedia applications such as virtual street roaming and 3D TV. It is also a challenging problem because the large translations and rotations of image patches make it hard to estimate the motion fields between wide-baseline image pairs. We propose a refinement strategy based on salient error detection to improve the result of existing approaches of wide-baseline image interpolation, where we combine the advantages of methods based on piecewise-linear transformation and methods based on variational model. We first use a lightweight interpolation method to estimate the initial motion field between the input image pair, and synthesize the intermediate image as the initial result. Then we detect regions with noticeable artifacts in the initial image to find areas whose motion vectors should be refined. Finally, we refine the motion field of the detected regions using a variational model based method, and obtain the refined intermediate image. The refinement strategy of our method can be used as the post refinement step for many other image interpolation algorithms. We show the effectiveness and efficiency of our method through experiments on different datasets.