Asymmetric Convolutional Residual Network for AV1 Intra in-Loop Filtering

In video compression standards, in-loop filtering plays an important role in alleviating blocking, blurring and ringing artifacts caused by lossy compression, which enhances visual quality and benefits coding efficiency. The boom of neural network applications in super-resolution and image restoration brings insights into solutions of in-loop filtering in video codecs. In this paper, we design an asymmetric convolutional residual network (ACRN) for in-loop filtering in the state-of-the-art AV1 codec. With the asymmetric convolutional blocks, directional features can be extracted to restore textures and improve quality. The cascading structure of wide-activated residual blocks with pruned dense connections enables reflecting hierarchical coding unit (CU) partition characteristics of video coding without losing overall details. Experiments show that the proposed lightweight ACRN can bring up to 12.78% coding efficiency improvement in intra coding of AV1.

[1]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Yue Chen,et al.  An Overview of Core Coding Tools in the AV1 Video Codec , 2018, 2018 Picture Coding Symposium (PCS).

[3]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Debargha Mukherjee,et al.  AV1 in-loop Filtering using a Wide-Activation Structured Residual Network , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[5]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Xinfeng Zhang,et al.  Spatial-temporal residue network based in-loop filter for video coding , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[7]  Xiaoyan Sun,et al.  Quality-Gated Convolutional Lstm for Enhancing Compressed Video , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Thomas S. Huang,et al.  Wide-activated Deep Residual Networks based Restoration for BPG-compressed Images , 2018, CVPR Workshops.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Qionghai Dai,et al.  Residual Highway Convolutional Neural Networks for in-loop Filtering in HEVC , 2018, IEEE Transactions on Image Processing.

[12]  Jungong Han,et al.  ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Dong Liu,et al.  A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding , 2016, MMM.

[14]  F. Bossen,et al.  Common test conditions and software reference configurations , 2010 .