A quality enhancement framework with noise distribution characteristics for high efficiency video coding

Abstract Video coding effectively reduces the amount of video data while unavoidably producing compression noise. Compression noise can cause significant artifacts in compressed video, such as blocking, ringing, and blurring, which seriously affects the visual quality of videos and the value of videos for content analysis. In compressed video quality enhancement, few methods based on deep learning fully consider the relationship between video content and compression noise or the possibility of uniting the encoder or the decoder to enhance the quality of compressed video. In an approach different from existing methods, we propose a video quality enhancement framework based on the distribution characteristics of compression noise. The proposed framework consists of two parts: at the encoder, we propose a convolutional neural network (CNN)-based in-loop filtering network combined with noise distribution (IFN-ND) characteristics for the I frame instead of high efficiency video coding (HEVC) standard in-loop filters; at the decoder, we propose a CNN-based quality enhancement network combined with the noise distribution characteristics (PQEN-ND) for the P frames. The noise characteristics are extracted from the code stream to further improve the performance of the proposed networks. The experiments show that the proposed method can significantly improve the quality of HEVC compressed video, achieving an average 12.84% reduction in the BD rate and up to a 1.0476 dB increase in the peak signal-to-noise ratio (PSNR).