Rotation invariant multi-frame image super resolution reconstruction using Pseudo Zernike Moments

The purpose of multi-frame super resolution (SR) is to combine multiple low resolution (LR) images to produce one high resolution (HR) image. The major challenge of classic SR approaches is accurate motion estimation between the frames. To address this problem, fuzzy motion estimation method has been proposed that replaces value of each pixel using the weighted average of all its neighboring pixels in all LR images. However, in case of rotation between LR images, comparing the gray level of blocks is not a suitable criterion for calculating the weight. Hence, magnitude of Zernike Moments (ZM) has been used as a rotation invariant feature. Considering the more robustness of Pseudo Zernike Moments (PZM) to noise and its higher description capability for the same order compared to ZM, in this paper, we propose a new method based on the magnitude of PZM as a rotation invariant descriptor for representing the pixels in the weight calculation. Also, due to the fact that the phase of PZM provides significant information for image reconstruction, we propose a new phase-based PZM descriptor for SR by making the phase coefficients invariant to rotation. Experimental results on several image sequences demonstrate that the proposed algorithm outperforms other currently popular SR techniques from the viewpoint of PSNR, SSIM and visual image quality. We propose multi-frame image SR based on magnitude and phase coefficients of PZM.Our method is rotation invariant and robust to image noise.Our results are satisfactory from the aspects of objective and subjective metrics.The proposed algorithm performs superior to the compared benchmark approaches.

[1]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[2]  Xuelong Li,et al.  Single-Image Super-Resolution via Sparse Coding Regression , 2011, 2011 Sixth International Conference on Image and Graphics.

[3]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[5]  Harry Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Xuelong Li,et al.  Image Quality Assessment Based on Multiscale Geometric Analysis , 2009, IEEE Transactions on Image Processing.

[7]  Lei Zhang,et al.  Centralized sparse representation for image restoration , 2011, 2011 International Conference on Computer Vision.

[8]  Xuelong Li,et al.  Single Image Super-Resolution With Multiscale Similarity Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[10]  Shan Li,et al.  Complex Zernike Moments Features for Shape-Based Image Retrieval , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[11]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[12]  George Wolberg,et al.  Digital image warping , 1990 .

[13]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[14]  Dit-Yan Yeung,et al.  Image Hallucination Using Neighbor Embedding over Visual Primitive Manifolds , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Gholamreza Anbarjafari,et al.  IMAGE Resolution Enhancement by Using Discrete and Stationary Wavelet Decomposition , 2011, IEEE Transactions on Image Processing.

[16]  Xuelong Li,et al.  Joint Learning for Single-Image Super-Resolution via a Coupled Constraint , 2012, IEEE Transactions on Image Processing.

[17]  J. D. van Ouwerkerk,et al.  Image super-resolution survey , 2006, Image Vis. Comput..

[18]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Stephen J. Roberts,et al.  Bayesian Image Super-resolution, Continued , 2006, NIPS.

[20]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[21]  Xuelong Li,et al.  Partially Supervised Neighbor Embedding for Example-Based Image Super-Resolution , 2011, IEEE Journal of Selected Topics in Signal Processing.

[22]  Il-hong Shin,et al.  Image Resolution Enhancement using Inter-Subband Correlation in Wavelet Domain , 2007, 2007 IEEE International Conference on Image Processing.

[23]  Wen-Yu Su,et al.  Recursive high-resolution reconstruction of blurred multiframe images , 1993, IEEE Trans. Image Process..

[24]  Nanning Zheng,et al.  Image hallucination with primal sketch priors , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Stanley Osher,et al.  Image Super-Resolution by TV-Regularization and Bregman Iteration , 2008, J. Sci. Comput..

[26]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[27]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[28]  Seunghyeon Rhee,et al.  Discrete cosine transform based regularized high-resolution image reconstruction algorithm , 1999 .

[29]  Jean-Michel Morel,et al.  Denoising image sequences does not require motion estimation , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[30]  Liangpei Zhang,et al.  A MAP Approach for Joint Motion Estimation, Segmentation, and Super Resolution , 2007, IEEE Transactions on Image Processing.

[31]  Hasan Demirel,et al.  Motion based video super resolution using edge directed interpolation and complex wavelet transform , 2013, Signal Process..

[32]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[33]  Thomas S. Mosley,et al.  Thomas S , 2005 .

[34]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[35]  Raveendran Paramesran,et al.  On the computational aspects of Zernike moments , 2007, Image Vis. Comput..

[36]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[37]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[38]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[39]  Shen Lijun,et al.  Image Super-Resolution Based on MCA and Wavelet-Domain HMT , 2010, 2010 International Forum on Information Technology and Applications.

[40]  Moshe Ben-Ezra,et al.  Penrose Pixels Super-Resolution in the Detector Layout Domain , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[41]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[42]  Shutao Li,et al.  Sparse representation with morphologic regularizations for single image super-resolution , 2014, Signal Process..

[43]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[45]  Roland T. Chin,et al.  On image analysis by the methods of moments , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  R. O. Lane,et al.  Non-parametric Bayesian super-resolution , 2010 .

[47]  Karim Faez,et al.  Face recognition using adaptively weighted patch PZM array from a single exemplar image per person , 2008, Pattern Recognit..

[48]  Xuelong Li,et al.  Zernike-Moment-Based Image Super Resolution , 2011, IEEE Transactions on Image Processing.

[49]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[50]  A. Bhatia,et al.  On the circle polynomials of Zernike and related orthogonal sets , 1954, Mathematical Proceedings of the Cambridge Philosophical Society.

[51]  Roland T. Chin,et al.  On Image Analysis by the Methods of Moments , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Truong Q. Nguyen,et al.  Markov Random Field Model-Based Edge-Directed Image Interpolation , 2007, IEEE Transactions on Image Processing.

[53]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[54]  J. Flusser,et al.  Moments and Moment Invariants in Pattern Recognition , 2009 .

[55]  Truong Q. Nguyen,et al.  Markov Random Field Model-Based Edge-Directed Image Interpolation , 2007, ICIP.