A Universal Encoder Rate Distortion Optimization Framework for Learned Compression

Learning-based image compression has drawn increasing attention in recent years. Despite impressive progress has been made, it still lacks a universal encoder optimization method to seek efficient representation for different images. In this paper, we develop a universal rate distortion optimization framework for learning-based compression, which adaptively optimizes latents and side information together for each image. The proposed framework is independent of network architecture and can be flexibly applied to existing and potential future compression networks. Experimental results demonstrate that we can achieve 6.6% bit rate saving against the latest traditional codec, i.e., VVC, yielding the state-of-the-art compression ratio. Moreover, with the proposed optimization framework, we win the first place in CLIC validation phase for all the three different bit rates in terms of PSNR.