Checkerboard Context Model for Efficient Learned Image Compression

For learned image compression, the autoregressive context model is proved effective in improving the rate-distortion (RD) performance. Because it helps remove spatial redundancies among latent representations. However, the decoding process must be done in a strict scan order, which breaks the parallelization. We propose a parallelizable checkerboard context model (CCM) to solve the problem. Our two-pass checkerboard context calculation eliminates such limitations on spatial locations by re-organizing the decoding order. Speeding up the decoding process more than 40 times in our experiments, it achieves significantly improved computational efficiency with almost the same rate-distortion performance. To the best of our knowledge, this is the first exploration on parallelization-friendly spatial context model for learned image compression.

[1]  Lucas Theis,et al.  Lossy Image Compression with Compressive Autoencoders , 2017, ICLR.

[2]  Luc Van Gool,et al.  Conditional Probability Models for Deep Image Compression , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Johannes Ballé,et al.  Efficient Nonlinear Transforms for Lossy Image Compression , 2018, 2018 Picture Coding Symposium (PCS).

[4]  David Minnen,et al.  Channel-Wise Autoregressive Entropy Models for Learned Image Compression , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[5]  Vivek K. Goyal,et al.  Theoretical foundations of transform coding , 2001, IEEE Signal Process. Mag..

[6]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Jooyoung Lee,et al.  An End-to-End Joint Learning Scheme of Image Compression and Quality Enhancement with Improved Entropy Minimization. , 2019 .

[8]  David Minnen,et al.  Full Resolution Image Compression with Recurrent Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Jing Zhou,et al.  Multi-scale and Context-adaptive Entropy Model for Image Compression , 2019, CVPR Workshops.

[10]  David Minnen,et al.  Joint Autoregressive and Hierarchical Priors for Learned Image Compression , 2018, NeurIPS.

[11]  David Zhang,et al.  Learning Convolutional Networks for Content-Weighted Image Compression , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Sihan Wen,et al.  Variational Autoencoder based Image Compression with Pyramidal Features and Context Entropy Model , 2019, CVPR Workshops.

[13]  Hui Yong Kim,et al.  Extended End-to-End optimized Image Compression Method based on a Context-Adaptive Entropy Model , 2019, CVPR Workshops.

[14]  Alexey Dosovitskiy,et al.  You Only Train Once: Loss-Conditional Training of Deep Networks , 2020, ICLR.

[15]  Luc Van Gool,et al.  Generative Adversarial Networks for Extreme Learned Image Compression , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Vladlen Koltun,et al.  Learning to Inpaint for Image Compression , 2017, NIPS.

[17]  Elad Eban,et al.  Computationally Efficient Neural Image Compression , 2019, ArXiv.

[18]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[19]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2013, The Kluwer international series in engineering and computer science.

[20]  Lei Zhou,et al.  End-to-end Optimized Image Compression with Attention Mechanism , 2019, CVPR Workshops.

[21]  David Minnen,et al.  Integer Networks for Data Compression with Latent-Variable Models , 2019, ICLR.

[22]  Jungwon Lee,et al.  Variable Rate Deep Image Compression With a Conditional Autoencoder , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Nicola Asuni,et al.  TESTIMAGES: a Large-scale Archive for Testing Visual Devices and Basic Image Processing Algorithms , 2014, STAG.

[24]  David Minnen,et al.  Variational image compression with a scale hyperprior , 2018, ICLR.

[25]  Jooyoung Lee,et al.  Context-adaptive Entropy Model for End-to-end Optimized Image Compression , 2018, ICLR.

[26]  G. Nigel Martin,et al.  * Range encoding: an algorithm for removing redundancy from a digitised message , 1979 .

[27]  Heiko Schwarz,et al.  Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[28]  Jiro Katto,et al.  Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  David Minnen,et al.  Image-Dependent Local Entropy Models for Learned Image Compression , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[30]  Glen G. Langdon,et al.  Universal modeling and coding , 1981, IEEE Trans. Inf. Theory.

[31]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[32]  Jane You,et al.  Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compression , 2019, IEEE Transactions on Image Processing.

[33]  Lubomir D. Bourdev,et al.  Real-Time Adaptive Image Compression , 2017, ICML.

[34]  Lei Zhou,et al.  Variational Autoencoder for Low Bit-rate Image Compression , 2018, CVPR Workshops.

[35]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[36]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[37]  Valero Laparra,et al.  End-to-end Optimized Image Compression , 2016, ICLR.