Nonlinear image interpolation via deep neural network

Model-based approaches toward image interpolation attempt to predict unknown pixels at the high-resolution (HR) from a given low-resolution (LR) image. In the past decade, various adaptive image interpolation methods have been develop aiming at better recovering important image structures such as edges and textures. However, those methods are all based on an implicit assumption about the linear relationship between LR and HR pixels partially due to the difficulty with modeling nonlinear relationship. In this paper, we propose to take an explicit learning-based approach toward modeling the nonlinear relationship between LR an HR pixels. A six-layer convolutional neural network with rectified linear units (ReLU) is presented and trained to learn the targeted nonlinear mapping from training data for image interpolation. Our experimental results have shown the proposed learning-based approach is often capable of achieving superior performance both subjectively and objectively to existing model-based methods.

[1]  P.J.L. van Beek,et al.  Edge-Based Image Representation and Coding , 1995 .

[2]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[3]  Wen Gao,et al.  Image interpolation via regularized local linear regression , 2011, 28th Picture Coding Symposium.

[4]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Rabab Kreidieh Ward,et al.  A New Orientation-Adaptive Interpolation Method , 2007, IEEE Transactions on Image Processing.

[6]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[7]  Hao Hu,et al.  Image interpolation using classification-based neural networks , 2004, IEEE International Symposium on Consumer Electronics, 2004.

[8]  Jiashi Feng,et al.  Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution. , 2017, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[9]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[10]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[11]  Wan-Chi Siu,et al.  Adaptive Directional Window Selection for Edge-Directed Interpolation , 2010, 2010 Proceedings of 19th International Conference on Computer Communications and Networks.

[12]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[13]  Xin Li,et al.  Guided deep network for depth map super-resolution: How much can color help? , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[15]  Mei-Juan Chen,et al.  A fast edge-oriented algorithm for image interpolation , 2005, Image Vis. Comput..

[16]  Jin-Jang Leou,et al.  Saliency-directed color image interpolation using artificial neural network and particle swarm optimization , 2012, J. Vis. Commun. Image Represent..

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[20]  Thierry Blu,et al.  Linear interpolation revitalized , 2004, IEEE Transactions on Image Processing.

[21]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Hsieh Hou,et al.  Cubic splines for image interpolation and digital filtering , 1978 .

[23]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[24]  Stephen M. Omohundro,et al.  Nonlinear Image Interpolation using Manifold Learning , 1994, NIPS.

[25]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[26]  Kwanghoon Sohn,et al.  Interpolation using neural networks for digital still cameras , 2000, IEEE Trans. Consumer Electron..

[27]  Truong Q. Nguyen,et al.  Markov Random Field Model-Based Edge-Directed Image Interpolation , 2007, IEEE Transactions on Image Processing.