Blind Multi-frame Super Resolution with Non-identical Blur

Real world video super resolution is an challenging problem due to the complex motion field and unknown blur kernel. Although multi-frame super resolution has been extensively studied in past decades, it still remained problems and always assumed that the blur kernels were identical in different frames. In this paper, we propose an novel blind multi-frame super resolution method with non-identical blur. To estimate blur kernels of different frames, we propose using salient edges selection method for more accurate kernel estimation. The whole process of estimation is based on Hyper-Laplacian prior, and iterative value updating through a multi-scale process. After the kernels of different frames are estimated, the high resolution frame is reconstructed using a cost function. The proposed method can obtain superior results, and outperforms the state of the art in the experiments through subjective and objective evaluation.

[1]  Mingkui Tan,et al.  Blind Image Deconvolution by Automatic Gradient Activation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yanning Zhang,et al.  Single Image Super-resolution Using Deformable Patches , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Dinesh Rajan,et al.  Unified Blind Method for Multi-Image Super-Resolution and Single/Multi-Image Blur Deconvolution , 2013, IEEE Transactions on Image Processing.

[4]  Deqing Sun,et al.  A Bayesian approach to adaptive video super resolution , 2011, CVPR 2011.

[5]  Ian D. Reid,et al.  From Motion Blur to Motion Flow: A Deep Learning Solution for Removing Heterogeneous Motion Blur , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ming-Hsuan Yang,et al.  Deblurring Text Images via L0-Regularized Intensity and Gradient Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Enhua Wu,et al.  Handling motion blur in multi-frame super-resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  L. Rudin,et al.  Feature-oriented image enhancement using shock filters , 1990 .

[9]  Chi-Keung Tang,et al.  Fast image/video upsampling , 2008, SIGGRAPH 2008.

[10]  Dinesh Rajan,et al.  Blind Super Resolution of Real-Life Video Sequences , 2016, IEEE Transactions on Image Processing.

[11]  Fan Guo-liang Overview on Super Resolution Image Reconstruction , 2011 .

[12]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[14]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, SIGGRAPH 2008.

[15]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[16]  Ming-Hsuan Yang,et al.  Robust Kernel Estimation with Outliers Handling for Image Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[18]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Horst Bischof,et al.  Conditioned Regression Models for Non-blind Single Image Super-Resolution , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[20]  Raanan Fattal Image upsampling via imposed edge statistics , 2007, SIGGRAPH 2007.

[21]  Deqing Sun,et al.  Blind Image Deblurring Using Dark Channel Prior , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[23]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[24]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[25]  Sunghyun Cho,et al.  Fast motion deblurring , 2009, SIGGRAPH 2009.