An MRF-Based DeInterlacing Algorithm With Exemplar-Based Refinement

In this paper, we propose an MRF-based deinterlacing algorithm that combines the benefits of rule-based algorithms such as motion-adaptation, edge-directed interpolation, and motion compensation, with those of an MRF formulation. MRF-based interpolation and enhancement algorithms are typically formulated as an optimization over pixel intensities or colors, which can make them relatively slow. In comparison, our MRF-based deinterlacing algorithm uses interpolation functions as labels. We use seven interpolants (three spatial, three temporal, and one for motion compensation). The core dynamic programming algorithm is, therefore, sped up greatly over the direct use of intensity as labels. We also show how an exemplar-based learning algorithm can be used to refine the output of our MRF-based algorithm. The training set can be augmented with exemplars from static regions of the same video, as a form of ldquoself-learningrdquo.

[1]  Luc Vandendorpe,et al.  Motion-compensated conversion from interlaced to progressive formats , 1994, Signal Process. Image Commun..

[2]  Luc Vandendorpe,et al.  Improved interpolation, motion estimation, and compensation for interlaced pictures , 1994, IEEE Trans. Image Process..

[3]  A. Murat Tekalp,et al.  Robust methods for high-quality stills from interlaced video in the presence of dominant motion , 1997, IEEE Trans. Circuits Syst. Video Technol..

[4]  Erwin B. Bellers,et al.  Deinterlacing-an overview , 1998, Proc. IEEE.

[5]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[6]  Gerard de Haan,et al.  IC for motion compensated de-interlacing, noise reduction, and picture rate conversion , 1999, 1999 Digest of Technical Papers. International Conference on Consumer Electronics (Cat. No.99CH36277).

[7]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  Yeong Ho Ha,et al.  Adaptive scan rate up-conversion system based on human visual characteristics , 2000, IEEE Trans. Consumer Electron..

[9]  Stan Z. Li,et al.  Markov Random Field Modeling in Image Analysis , 2001, Computer Science Workbench.

[10]  Seungjoon Yang,et al.  Motion compensation assisted motion adaptive interlaced-to-progressive conversion , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[11]  E. Shechtman,et al.  Transactions on Pattern Analysis and Machine Intelligence 1 Space-time Video Completion Draft Transactions on Pattern Analysis and Machine Intelligence 2 , 2022 .

[12]  Hanna M. Wallach,et al.  Conditional Random Fields: An Introduction , 2004 .

[13]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Ming Gu,et al.  Advanced Motion Search and Adaptation Techniques for Deinterlacing , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[15]  Liang-Gee Chen,et al.  Video de-interlacing by adaptive 4-field global/local motion compensated approach , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Demin Wang,et al.  Hybrid de-interlacing algorithm based on motion vector reliability , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Truong Q. Nguyen,et al.  Performance analysis of motion-compensated de-interlacing systems , 2006, IEEE Transactions on Image Processing.

[18]  Truong Q. Nguyen,et al.  A De-Interlacing Algorithm Using Markov Random Field Model , 2007, IEEE Transactions on Image Processing.

[19]  Gwo Giun Lee,et al.  Motion Adaptive Deinterlacing via Edge Pattern Recognition , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[20]  de G Gerard Haan,et al.  Making the best of legacy video on modern displays , 2007 .