Frame interpolation with pixel-level motion vector field and mesh based hole filling

Abstract Most of the traditional methods are based on block motion compensation tending to involve heavy blocking artifacts in the interpolated frames. In this paper, a new frame interpolation method with pixel-level motion vector field (MVF) is proposed. Our method consists of the following four steps: (i) applying the pixel-level motion vectors (MVs) estimated by optical flow algorithm to eliminate blocking artifacts (ii) motion post-processing and super-sampling anti-aliasing to solve the problems caused by pixel-level MVs (iii) robust warping method to address collisions and holes caused by occlusions (iv) a new holes filling method using triangular mesh (HFTM) to reduce the artifacts caused by holes. Experimental results show that the proposed method can effectively alleviate the holes and blocking artifacts in interpolated frames, and outperforms existing methods both in terms of objective and subjective performances, especially for sequences with complex motions.