True motion compensation with feature detection for frame rate up-conversion

This paper presents a feature-based frame rate up-conversion algorithm which provides more comfortable visual experience by exploiting true motion of the objects. By considering the movement of the objects rather than the pixel values, the proposed method can create interpolated frames to reflect true movement of the video contents. We first find local features within a frame by using a feature detection algorithm. Then, the local features are matched between adjacent frames and are clustered to form an object region. The interpolated frame is created by using the perspective transformation, which enables to adequately track the dynamic movement of the defined objects. The proposed scheme efficiently resolves the blocking artifact problem and presents outstanding visual quality compared to the conventional block-based motion compensated interpolation algorithm.

[1]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[2]  Tobias Höllerer,et al.  Evaluation of Interest Point Detectors and Feature Descriptors for Visual Tracking , 2011, International Journal of Computer Vision.

[3]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[4]  Sungjoo Yoo,et al.  Dual Motion Estimation for Frame Rate Up-Conversion , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Demin Wang,et al.  Motion-Compensated Frame Rate Up-Conversion—Part II: New Algorithms for Frame Interpolation , 2010, IEEE Transactions on Broadcasting.

[6]  Xuran Zhao,et al.  A subspace co-training framework for multi-view clustering , 2014, Pattern Recognit. Lett..

[7]  Patrick Pérez,et al.  Dense estimation and object-based segmentation of the optical flow with robust techniques , 1998, IEEE Trans. Image Process..

[8]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[9]  Bart Braden.  The Surveyor's Area Formula , 1986 .

[10]  Truong Q. Nguyen,et al.  Motion vector processing for frame rate up conversion , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Sun-Yuan Kung,et al.  Frame-rate up-conversion using transmitted true motion vectors , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[12]  Chung-Lin Huang,et al.  Motion‐compensated interpolation for scan rate up‐conversion , 1996 .

[13]  Roberto Castagno,et al.  A method for motion adaptive frame rate up-conversion , 1996, IEEE Trans. Circuits Syst. Video Technol..

[14]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[15]  Sung-Jea Ko,et al.  New frame rate up-conversion using bi-directional motion estimation , 2000, IEEE Trans. Consumer Electron..

[16]  Jenny Benois-Pineau,et al.  A New Method for Region-Based Depth Ordering in a Video Sequence: Application to Frame Interpolation , 2002, J. Vis. Commun. Image Represent..

[17]  Rae-Hong Park,et al.  Weighted-adaptive motion-compensated frame rate up-conversion , 2003, IEEE Trans. Consumer Electron..

[18]  Sunil Arya,et al.  Approximate nearest neighbor queries in fixed dimensions , 1993, SODA '93.

[19]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[20]  Kyoung-Rok Cho,et al.  Motion Compensated Frame Rate Up-Conversion Using Extended Bilateral Motion Estimation , 2007, IEEE Transactions on Consumer Electronics.

[21]  Deokho Kim,et al.  Dynamic Load Balancing of Parallel SURF with Vertical Partitioning , 2015, IEEE Transactions on Parallel and Distributed Systems.

[22]  Demin Wang,et al.  Motion-Compensated Frame Rate Up-Conversion—Part I: Fast Multi-Frame Motion Estimation , 2010, IEEE Transactions on Broadcasting.