Global motion based video super-resolution reconstruction using discrete wavelet transform

Different from the existing super-resolution (SR) reconstruction approaches working under either the frequency-domain or the spatial- domain, this paper proposes an improved video SR approach based on both frequency and spatial-domains to improve the spatial resolution and recover the noiseless high-frequency components of the observed noisy low-resolution video sequences with global motion. An iterative planar motion estimation algorithm followed by a structure-adaptive normalised convolution reconstruction method are applied to produce the estimated low-frequency sub-band. The discrete wavelet transform process is employed to decompose the input low-resolution reference frame into four sub-bands, and then the new edge-directed interpolation method is used to interpolate each of the high-frequency sub-bands. The novelty of this algorithm is the introduction and integration of a nonlinear soft thresholding process to filter the estimated high-frequency sub-bands in order to better preserve the edges and remove potential noise. Another novelty of this algorithm is to provide flexibility with various motion levels, noise levels, wavelet functions, and the number of used low-resolution frames. The performance of the proposed method has been tested on three well-known videos. Both visual and quantitative results demonstrate the high performance and improved flexibility of the proposed technique over the conventional interpolation and the state-of-the-art video SR techniques in the wavelet- domain.

[1]  Daniel Gross,et al.  Improved resolution from subpixel shifted pictures , 1992, CVGIP Graph. Model. Image Process..

[2]  Hamid Gharavi Editorial Message From the Incoming Editor-in-Chief , 2010, IEEE Trans. Circuits Syst. Video Technol..

[3]  Liangpei Zhang,et al.  A super-resolution reconstruction algorithm for surveillance images , 2010, Signal Process..

[4]  Gholamreza Anbarjafari,et al.  Video resolution enhancement by using discrete and stationary wavelet transforms with illumination compensation , 2015, Signal Image Video Process..

[5]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[6]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[7]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[8]  Shih-Chia Huang,et al.  An Advanced Motion Detection Algorithm With Video Quality Analysis for Video Surveillance Systems , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[10]  Yifan Zhao,et al.  An Optimal Factor Analysis Approach to Improve the Wavelet-based Image Resolution Enhancement Techniques , 2016 .

[11]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[12]  Shmuel Peleg,et al.  Improving image resolution using subpixel motion , 1987, Pattern Recognit. Lett..

[13]  B. Marcel,et al.  3 - Calcul de translation et rotation par la transformation de Fourier , 1997 .

[14]  Xiao-Ping Zhang,et al.  Thresholding neural network for adaptive noise reduction , 2001, IEEE Trans. Neural Networks.

[15]  A. Murat Tekalp,et al.  Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time , 1997, IEEE Trans. Image Process..

[16]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[17]  Klamer Schutte,et al.  Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution , 2006, EURASIP J. Adv. Signal Process..

[18]  Hayit Greenspan,et al.  Super-Resolution in Medical Imaging , 2009, Comput. J..

[19]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[20]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[21]  Alptekin Temizel Image Resolution Enhancement using Wavelet Domain Hidden Markov Tree and Coefficient Sign Estimation , 2007, 2007 IEEE International Conference on Image Processing.

[22]  Robert L. Stevenson,et al.  A Bayesian approach to image expansion for improved definitio , 1994, IEEE Trans. Image Process..

[23]  Peyman Milanfar,et al.  INTERPOLATION-RESTORATION METHOD FOR SUPERRESOLUTION (WAVELET SUPERRESOLUTION)* , 2000 .

[24]  Shiguang Shan,et al.  Deep Network Cascade for Image Super-resolution , 2014, ECCV.

[25]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Yongdong Zhang,et al.  A Highly Parallel Framework for HEVC Coding Unit Partitioning Tree Decision on Many-core Processors , 2014, IEEE Signal Processing Letters.

[28]  Abdul Ghafoor,et al.  Satellite Image Resolution Enhancement Using Dual-Tree Complex Wavelet Transform and Nonlocal Means , 2013, IEEE Geoscience and Remote Sensing Letters.

[29]  H. Demirel,et al.  Image Super Resolution Based on Interpolation of Wavelet Domain High Frequency Subbands and the Spatial Domain Input Image , 2010 .

[30]  Nikolas P. Galatsanos,et al.  Reconstruction of a high resolution image from registration and restoration of low resolution images , 1994, Proceedings of 1st International Conference on Image Processing.

[31]  Michel Barlaud,et al.  Image coding using wavelet transform , 1992, IEEE Trans. Image Process..

[32]  Thomas S. Huang,et al.  Self-tuned deep super resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Sabine Süsstrunk,et al.  A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution , 2006, EURASIP J. Adv. Signal Process..

[34]  Kun Li,et al.  Video super-resolution using an adaptive superpixel-guided auto-regressive model , 2016, Pattern Recognit..

[35]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[37]  Ping Wah Wong,et al.  Edge-directed interpolation , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[38]  Kai-Kuang Ma,et al.  A survey on super-resolution imaging , 2011, Signal Image Video Process..

[39]  Gholamreza Anbarjafari,et al.  DWT Based Resolution Enhancement of Video Sequences , 2013 .

[40]  Luca Lucchese,et al.  A noise-robust frequency domain technique for estimating planar roto-translations , 2000, IEEE Trans. Signal Process..

[41]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Joseph Y. Lo,et al.  New Applications of Super-Resolution in Medical Imaging , 2017 .

[43]  Michael Elad,et al.  Super-Resolution Without Explicit Subpixel Motion Estimation , 2009, IEEE Transactions on Image Processing.

[44]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[45]  Jayanthi Pragatheeswaran,et al.  Image resolution enhancement based on edge directed interpolation using dual tree — Complex wavelet transform , 2011, 2011 International Conference on Recent Trends in Information Technology (ICRTIT).

[46]  Yongdong Zhang,et al.  Effective Uyghur Language Text Detection in Complex Background Images for Traffic Prompt Identification , 2018, IEEE Transactions on Intelligent Transportation Systems.

[47]  Paulo Lobato Correia,et al.  Super-resolution of facial images in forensics scenarios , 2015, 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA).

[48]  Sergios Theodoridis,et al.  A Novel Efficient Cluster-Based MLSE Equalizer for Satellite Communication Channels with-QAM Signaling , 2006, EURASIP J. Adv. Signal Process..

[49]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[50]  A. Murat Tekalp,et al.  Robust, object-based high-resolution image reconstruction from low-resolution video , 1997, IEEE Trans. Image Process..

[51]  Gholamreza Anbarjafari,et al.  Discrete Wavelet Transform-Based Satellite Image Resolution Enhancement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[53]  Gholamreza Anbarjafari,et al.  IMAGE Resolution Enhancement by Using Discrete and Stationary Wavelet Decomposition , 2011, IEEE Transactions on Image Processing.

[54]  Zhuowen Tu,et al.  Robust $L_{2}E$ Estimation of Transformation for Non-Rigid Registration , 2015, IEEE Transactions on Signal Processing.

[55]  Hasan Demirel,et al.  Motion based video super resolution using edge directed interpolation and complex wavelet transform , 2013, Signal Process..

[56]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[57]  G. Singh,et al.  Cuckoo search algorithm based satellite image contrast and brightness enhancement using DWT-SVD. , 2014, ISA transactions.

[58]  Yongdong Zhang,et al.  Efficient Parallel Framework for HEVC Motion Estimation on Many-Core Processors , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[59]  Renjie Liao,et al.  Video Super-Resolution via Deep Draft-Ensemble Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[60]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[61]  Ruimin Hu,et al.  Efficient single image super-resolution via graph-constrained least squares regression , 2013, Multimedia Tools and Applications.

[62]  J. Flusser,et al.  PIZZARO: Forensic analysis and restoration of image and video data. , 2016, Forensic science international.

[63]  Hasan Demirel,et al.  Multi-frame super resolution using edge directed interpolation and complex wavelet transform , 2012 .

[64]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[65]  Shmuel Peleg,et al.  Image sequence enhancement using sub-pixel displacements , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[66]  Gholamreza Anbarjafari,et al.  Satellite Image Resolution Enhancement Using Complex Wavelet Transform , 2010, IEEE Geoscience and Remote Sensing Letters.

[67]  Yongdong Zhang,et al.  Supervised Hash Coding With Deep Neural Network for Environment Perception of Intelligent Vehicles , 2018, IEEE Transactions on Intelligent Transportation Systems.