Space-time super-resolution with patch group cuts prior

We address space-time super-resolution (SR) problem in this paper. To efficiently explore the correlations within video sequences, a patch group (PG) model is proposed. The model is based on a novel 3D neighborhood system (NS) and embeds the spatial and temporal correlations. A patch group cuts (PGCuts) metric is then built on the model to provide a new prior for space-time SR reconstruction. To balance the prior strength for the whole video sequence, an adaptive scheme is also considered. We evaluate the proposed PGCuts prior on both synthesized and real video sequences. The results indicate that space-time SR method with the PGCuts prior outperforms other ones by retaining smooth edges and motion trajectories in videos, and by staying robust to noises as well. 3D neighborhood system accommodates more pixel correlations than traditional one.Patch group model based on similarity efficiently exploits space-time coherence.Smooth surfaces in patch group model results in smooth edges and motion trajectories.Edge strength based adaptive scheme effectively balances performance in the whole video.SR with patch group cuts produces better objective and subjective performance.

[1]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Ying Wu,et al.  Super-Resolution Without Dense Flow , 2012, IEEE Transactions on Image Processing.

[3]  Thomas S. Huang,et al.  Image and Video Restorations via Nonlocal Kernel Regression , 2013, IEEE Transactions on Cybernetics.

[4]  Vladimir Kolmogorov,et al.  Computing geodesics and minimal surfaces via graph cuts , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[6]  Tomio Goto,et al.  Super resolution image reconstruction using total variation regularization and learning-based method , 2010, 2010 IEEE International Conference on Image Processing.

[7]  Mei Han,et al.  SoftCuts: A Soft Edge Smoothness Prior for Color Image Super-Resolution , 2009, IEEE Transactions on Image Processing.

[8]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Stanley Osher,et al.  Image Super-Resolution by TV-Regularization and Bregman Iteration , 2008, J. Sci. Comput..

[10]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[11]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[12]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[13]  Yaron Caspi,et al.  Under the supervision of , 2003 .

[14]  Michael Elad,et al.  Super-Resolution Without Explicit Subpixel Motion Estimation , 2009, IEEE Transactions on Image Processing.

[15]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[16]  Shree K. Nayar,et al.  Video super-resolution using controlled subpixel detector shifts , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Dinesh Rajan,et al.  Space-time super-resolution from multiple-videos , 2012, 2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA).

[20]  Subhashis Banerjee,et al.  Space-Time Super-Resolution Using Graph-Cut Optimization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Harry Shum,et al.  Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement , 2011, IEEE Transactions on Image Processing.

[23]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Yu-Chiang Frank Wang,et al.  A Self-Learning Approach to Single Image Super-Resolution , 2013, IEEE Transactions on Multimedia.

[25]  Xiaohai He,et al.  Adaptive regularization-based space-time super-resolution reconstruction , 2013, Signal Process. Image Commun..

[26]  Kai Chen,et al.  Image super-resolution based on a novel edge sharpness prior , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[27]  Michal Irani,et al.  Space-time super-resolution from a single video , 2011, CVPR 2011.

[28]  Jin Chen,et al.  Video Super-Resolution Using Generalized Gaussian Markov Random Fields , 2012, IEEE Signal Processing Letters.