Detail enhancement of image super-resolution based on detail synthesis

Abstract In this paper, a detail-enhancement and super-resolution algorithm based on detail synthesis is proposed. The novelty of this algorithm is in combining local self-similarity search and singular value decomposition of patches together to obtain details with more natural high-frequency. The proposed algorithm improves the facet or line phenomenon on edges and areas that have rich texture. The algorithm firstly searches for an image patch and extracts the high-frequency components based on a local self-similarity of the original, low-resolution image. The matrix of the high-frequency block is then decomposed into two sub-spaces by the singular value decomposition and the pseudo high-frequency is removed by a soft threshold. Then, the high-frequency block is reconstructed using effective singular values. The final super-resolution image is restored by the detail synthesis with the initial super-resolution image. The experimental results show that the proposed method can significantly remove the artificial effect of facet or line phenomenon caused by pseudo high-frequency. Moreover, the method is also applicable to other super-resolution algorithm in detail enhancement.

[1]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[2]  Wenhan Yang,et al.  Image Super-Resolution Based on Structure-Modulated Sparse Representation , 2015, IEEE Transactions on Image Processing.

[3]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Yi-Nung Liu,et al.  Algorithm and Architecture Design of High-Quality Video Upscaling Using Database-Free Texture Synthesis , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Jong Beom Ra,et al.  Example-Based Super-Resolution via Structure Analysis of Patches , 2013, IEEE Signal Processing Letters.

[7]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Gui-Song Xia,et al.  Learning High-level Features for Satellite Image Classification With Limited Labeled Samples , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Raanan Fattal,et al.  Image upsampling via texture hallucination , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[10]  Lei He,et al.  Image Super-Resolution Based on Sparse Representation , 2013 .

[11]  Pascal Getreuer,et al.  Contour Stencils: Total Variation along Curves for Adaptive Image Interpolation , 2011, SIAM J. Imaging Sci..

[12]  Robert Sablatnig,et al.  Comparison of reconstruction and example-based Super-Resolution , 2011, 2011 7th International Conference on Emerging Technologies.

[13]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[14]  Chandra Shaker Balure,et al.  A Survey -- Super Resolution Techniques for Multiple, Single, and Stereo Images , 2014, 2014 Fifth International Symposium on Electronic System Design.

[15]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[16]  Hongyuan Zha,et al.  Multiscale Semilocal Interpolation With Antialiasing , 2012, IEEE Transactions on Image Processing.

[17]  Qingquan Li,et al.  Multi-focus image fusion based on depth extraction with inhomogeneous diffusion equation , 2016, Signal Process..

[18]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Wu Chengdong,et al.  Overview on image super resolution reconstruction , 2014, The 26th Chinese Control and Decision Conference (2014 CCDC).

[20]  Soo-Chang Pei,et al.  Viewing-Distance Aware Super-Resolution for High-Definition Display , 2015, IEEE Transactions on Image Processing.

[21]  Michael Unser,et al.  Image interpolation and resampling , 2000 .

[22]  Shang-Hong Lai,et al.  Single Image Super-Resolution Based on Local Self-Similarity , 2013, 2013 2nd IAPR Asian Conference on Pattern Recognition.

[23]  Yuchen Yan,et al.  Adaptive shock filter for image super-resolution and enhancement , 2016, J. Vis. Commun. Image Represent..

[24]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[25]  Xiaokang Yang,et al.  Learning-based super-resolution method with a combining of both global and local constraints , 2012 .

[26]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[27]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Rogério Schmidt Feris,et al.  Edge guided single depth image super resolution , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[29]  Noriaki Suetake,et al.  Image super-resolution based on local self-similarity , 2008 .

[30]  Lei Zhang,et al.  Nonlocal back-projection for adaptive image enlargement , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[31]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[32]  Fabrizio Argenti,et al.  Comparison of super-resolution methods for quality enhancement of digital biomedical images , 2014, 2014 8th International Symposium on Medical Information and Communication Technology (ISMICT).

[33]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[34]  Yong-Qin Zhang.  Visibility enhancement using an image filtering approach , 2012, EURASIP J. Adv. Signal Process..

[35]  Chih-Yuan Yang,et al.  Exploiting Self-similarities for Single Frame Super-Resolution , 2010, ACCV.

[36]  Li Yi,et al.  Image Super-Resolution Via Analysis Sparse Prior , 2013, IEEE Signal Processing Letters.

[37]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[39]  Hsieh Hou,et al.  Cubic splines for image interpolation and digital filtering , 1978 .

[40]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.

[41]  Zhiyong Gao,et al.  Image super-resolution with facet improvement and detail enhancement based on local self examples , 2013, 2013 International Conference on Wireless Communications and Signal Processing.

[42]  Marc Moonen,et al.  Joint DOA and multi-pitch estimation based on subspace techniques , 2012, EURASIP J. Adv. Signal Process..

[43]  Jiejie Zhu,et al.  Context-constrained hallucination for image super-resolution , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Gaofeng Meng,et al.  Edge-Directed Single-Image Super-Resolution Via Adaptive Gradient Magnitude Self-Interpolation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.