D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images

In this paper, we design a Deep Dual-Domain (D3) based fast restoration model to remove artifacts of JPEG compressed images. It leverages the large learning capacity of deep networks, as well as the problem-specific expertise that was hardly incorporated in the past design of deep architectures. For the latter, we take into consideration both the prior knowledge of the JPEG compression scheme, and the successful practice of the sparsity-based dual-domain approach. We further design the One-Step Sparse Inference (1-SI) module, as an efficient and lightweighted feed-forward approximation of sparse coding. Extensive experiments verify the superiority of the proposed D3 model over several state-of-the-art methods. Specifically, our best model is capable of outperforming the latest deep model for around 1 dB in PSNR, and is 30 times faster.

[1]  Joan L. Mitchell,et al.  JPEG: Still Image Data Compression Standard , 1992 .

[2]  C.-C. Jay Kuo,et al.  Real-time compression artifact reduction via robust nonlinear filtering , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[3]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[4]  Taejeong Kim,et al.  Regression-based prediction for blocking artifact reduction in JPEG-compressed images , 2005, IEEE Transactions on Image Processing.

[5]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[6]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[7]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[8]  Karen O. Egiazarian,et al.  Pointwise Shape-Adaptive DCT for High-Quality Denoising and Deblocking of Grayscale and Color Images , 2007, IEEE Transactions on Image Processing.

[9]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[10]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[11]  Inderjit S. Dhillon,et al.  Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach , 2010, SIAM J. Sci. Comput..

[12]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Changhoon Yim,et al.  Quality Assessment of Deblocked Images , 2011, IEEE Transactions on Image Processing.

[14]  Kristian Bredies,et al.  A Total Variation-Based JPEG Decompression Model , 2012, SIAM J. Imaging Sci..

[15]  Sebastian Nowozin,et al.  Loss-Specific Training of Non-Parametric Image Restoration Models: A New State of the Art , 2012, ECCV.

[16]  Misha Denil,et al.  Recklessly Approximate Sparse Coding , 2012, ArXiv.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Nauman Atique,et al.  Extending the UCP Model by Incorporating the Prevailing Trends in Software Effort Estimation , 2012 .

[19]  Sanjay B. Dhok,et al.  Review of Proposed High Efficiency Video Coding (HEVC) Standard , 2012 .

[20]  Licheng Jiao,et al.  Image deblocking via sparse representation , 2012, Signal Process. Image Commun..

[21]  Guillermo Sapiro,et al.  Supervised Sparse Analysis and Synthesis Operators , 2013, NIPS.

[22]  Michael S. Brown,et al.  A Learning-Based Approach to Reduce JPEG Artifacts in Image Matting , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Anders P. Eriksson,et al.  Fast Convolutional Sparse Coding , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[25]  Jonathan Le Roux,et al.  Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures , 2014, ArXiv.

[26]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[27]  Michael K. Ng,et al.  Reducing Artifacts in JPEG Decompression Via a Learned Dictionary , 2014, IEEE Transactions on Signal Processing.

[28]  Lei Zhang,et al.  Projective dictionary pair learning for pattern classification , 2014, NIPS.

[29]  Xiaoou Tang,et al.  Compression Artifacts Reduction by a Deep Convolutional Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Xianming Liu,et al.  Inter-block consistent soft decoding of JPEG images with sparsity and graph-signal smoothness priors , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[31]  Guillermo Sapiro,et al.  Learning Efficient Sparse and Low Rank Models , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Xianming Liu,et al.  Data-driven sparsity-based restoration of JPEG-compressed images in dual transform-pixel domain , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Thomas S. Huang,et al.  Deeply Improved Sparse Coding for Image Super-Resolution , 2015, ArXiv.

[36]  Thomas S. Huang,et al.  Self-tuned deep super resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[37]  Luc Van Gool,et al.  Efficient regression priors for reducing image compression artifacts , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[38]  Jiayu Zhou,et al.  Learning A Task-Specific Deep Architecture For Clustering , 2015, SDM.

[39]  Qing Ling,et al.  Learning Deep $\ell_0$ Encoders , 2015, 1509.00153.

[40]  R. Sarpong,et al.  Bio-inspired synthesis of xishacorenes A, B, and C, and a new congener from fuscol† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c9sc02572c , 2019, Chemical science.