Learning a Virtual Codec Based on Deep Convolutional Neural Network to Compress Image

Although deep convolutional neural network has been proved to efficiently eliminate coding artifacts caused by the coarse quantization of traditional codec, it's difficult to train any neural network in front of the encoder for gradient's back-propagation. In this paper, we propose an end-to-end image compression framework based on convolutional neural network to resolve the problem of non-differentiability of the quantization function in the standard codec. First, the feature description neural network is used to get a valid description in the low-dimension space with respect to the ground-truth image so that the amount of image data is greatly reduced for storage or transmission. After image's valid description, standard image codec such as JPEG is leveraged to further compress image, which leads to image's great distortion and compression artifacts, especially blocking artifacts, detail missing, blurring, and ringing artifacts. Then, we use a post-processing neural network to remove these artifacts. Due to the challenge of directly learning a non-linear function for a standard codec based on convolutional neural network, we propose to learn a virtual codec neural network to approximate the projection from the valid description image to the post-processed compressed image, so that the gradient could be efficiently back-propagated from the post-processing neural network to the feature description neural network during training. Meanwhile, an advanced learning algorithm is proposed to train our deep neural networks for compression. Obviously, the priority of the proposed method is compatible with standard existing codecs and our learning strategy can be easily extended into these codecs based on convolutional neural network. Experimental results have demonstrated the advances of the proposed method as compared to several state-of-the-art approaches, especially at very low bit-rate.

[1]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[2]  Deqing Sun,et al.  Postprocessing of Low Bit-Rate Block DCT Coded Images Based on a Fields of Experts Prior , 2007, IEEE Transactions on Image Processing.

[3]  Qionghai Dai,et al.  Residual Highway Convolutional Neural Networks for in-loop Filtering in HEVC , 2018, IEEE Transactions on Image Processing.

[4]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[5]  Valero Laparra,et al.  End-to-end Optimized Image Compression , 2016, ICLR.

[6]  Wen Gao,et al.  CONCOLOR: Constrained Non-Convex Low-Rank Model for Image Deblocking , 2016, IEEE Transactions on Image Processing.

[7]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Hongyang Chao,et al.  Building Dual-Domain Representations for Compression Artifacts Reduction , 2016, ECCV.

[9]  Christine Guillemot,et al.  Real-time constrained TCP-compatible rate control for video over the Internet , 2004, IEEE Transactions on Multimedia.

[10]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Hong Yan,et al.  Blocking artifact reduction in compressed images based on edge-adaptive quadrangle meshes , 2003, J. Vis. Commun. Image Represent..

[12]  Luca Benini,et al.  CAS-CNN: A deep convolutional neural network for image compression artifact suppression , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[13]  Hong Yan,et al.  Blocking artifacts suppression in block-coded images using overcomplete wavelet representation , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Yao Zhao,et al.  Deep Multiple Description Coding by Learning Scalar Quantization , 2018, 2019 Data Compression Conference (DCC).

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Bo Yan,et al.  An efficient deep convolutional neural networks model for compressed image deblocking , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[17]  Yao Zhao,et al.  Simultaneous color-depth super-resolution with conditional generative adversarial networks , 2019, Pattern Recognit..

[18]  Jani Lainema,et al.  Adaptive deblocking filter , 2003, IEEE Trans. Circuits Syst. Video Technol..

[19]  Li-Wei Kang,et al.  Self-Learning Based Image Decomposition With Applications to Single Image Denoising , 2014, IEEE Transactions on Multimedia.

[20]  Jun Zhou,et al.  Adaptive non-local means filtering for image deblocking , 2011, 2011 4th International Congress on Image and Signal Processing.

[21]  Yao Zhao,et al.  Iterative range-domain weighted filter for structural preserving image smoothing and de-noising , 2017, Multimedia Tools and Applications.

[22]  Xiaolin Wu,et al.  Data-Driven Soft Decoding of Compressed Images in Dual Transform-Pixel Domain. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[23]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[24]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Michael Elad,et al.  Optimized Pre-Compensating Compression , 2017, IEEE Transactions on Image Processing.

[26]  Kunal N. Chaudhury,et al.  Fast Adaptive Bilateral Filtering , 2018, IEEE Transactions on Image Processing.

[27]  Gary J. Sullivan,et al.  Introduction to the Special Issue on HEVC Extensions and Efficient HEVC Implementations , 2016, IEEE Trans. Circuits Syst. Video Technol..

[28]  B. Zeng,et al.  Candidate value-based boundary filtering for compressed depth images , 2015 .

[29]  David Minnen,et al.  Variable Rate Image Compression with Recurrent Neural Networks , 2015, ICLR.

[30]  Yao Zhao,et al.  Simultaneously Color-Depth Super-Resolution with Conditional Generative Adversarial Network , 2017, ArXiv.

[31]  Karen O. Egiazarian,et al.  Pointwise Shape-Adaptive DCT for High-Quality Denoising and Deblocking of Grayscale and Color Images , 2007, IEEE Transactions on Image Processing.

[32]  Michael W. Marcellin,et al.  An overview of JPEG-2000 , 2000, Proceedings DCC 2000. Data Compression Conference.

[33]  David Minnen,et al.  Full Resolution Image Compression with Recurrent Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Luca Benini,et al.  Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations , 2017, NIPS.

[35]  Zulin Wang,et al.  Enhancing Quality for HEVC Compressed Videos , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Sushanta Gogoi,et al.  Design and Implementation of Efficient Streaming Deblocking and SAO Filter for HEVC Decoder , 2018, IEEE Transactions on Consumer Electronics.

[38]  Wen Gao,et al.  Compression Artifact Reduction by Overlapped-Block Transform Coefficient Estimation With Block Similarity , 2013, IEEE Transactions on Image Processing.

[39]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Gerard de Haan,et al.  Quality adaptive least squares trained filters for video compression artifacts removal using a no-reference block visibility metric , 2011, J. Vis. Commun. Image Represent..

[41]  Chaitali Chakrabarti,et al.  A high-performance JPEG2000 architecture , 2003, IEEE Trans. Circuits Syst. Video Technol..

[42]  Michael S. Brown,et al.  A Contrast Enhancement Framework with JPEG Artifacts Suppression , 2014, ECCV.

[43]  Xiaoou Tang,et al.  Compression Artifacts Reduction by a Deep Convolutional Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[44]  Alberto Del Bimbo,et al.  Deep Generative Adversarial Compression Artifact Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Ling Li,et al.  Adaptive deblocking method for low bitrate coded HEVC video , 2016, J. Vis. Commun. Image Represent..

[46]  Jae S. Lim,et al.  Reduction of blocking effect in image coding , 1983, ICASSP.

[47]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[48]  Wuzhen Shi,et al.  An End-to-End Compression Framework Based on Convolutional Neural Networks , 2017, 2017 Data Compression Conference (DCC).

[49]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Michael K. Ng,et al.  Reducing Artifacts in JPEG Decompression Via a Learned Dictionary , 2014, IEEE Transactions on Signal Processing.

[51]  Yiyan Wu,et al.  Media Transmission by Cooperation of Cellular Network and Broadcasting Network , 2017, IEEE Transactions on Broadcasting.

[52]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Hongliang Li,et al.  A Fast HEVC Inter CU Selection Method Based on Pyramid Motion Divergence , 2014, IEEE Transactions on Multimedia.

[54]  Yao Zhao,et al.  Multiple Description Convolutional Neural Networks for Image Compression , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[55]  Eduardo A. B. da Silva,et al.  A generic post-deblocking filter for block based image compression algorithms , 2012, Signal Process. Image Commun..

[56]  Xinpeng Zhang,et al.  An Effective CU Size Decision Method for HEVC Encoders , 2013, IEEE Transactions on Multimedia.

[57]  Qing Ling,et al.  D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Lucas Theis,et al.  Lossy Image Compression with Compressive Autoencoders , 2017, ICLR.

[59]  Jong Beom Ra,et al.  Post-Processing for Blocking Artifact Reduction Based on Inter-Block Correlation , 2014, IEEE Transactions on Multimedia.

[60]  David Zhang,et al.  Learning Convolutional Networks for Content-Weighted Image Compression , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.