$\lambda$-Domain Perceptual Rate Control for 360-Degree Video Compression

The 360-degree video is projected to 2-D formats using various projection methods for efficient compression. As a necessary part of general-video compression, rate control is also indispensable for the projected 360-degree video compression. However, the current rate control algorithm has not been optimized for the 360-degree video compression yet. The Coding Tree Unit (CTU) level bit allocation in the rate control algorithm has not taken into consideration the characteristic that various pixels in 2-D formats have different influences on the visual experiences. In this article, we first propose an optimal CTU level weight taking this characteristic into consideration. The CTU level weight is an approximation to the pixel level weight since the smallest granularity of a rate control algorithm is usually CTU. Second, based on the CTU level weight, a weighted CTU level bit allocation algorithm is proposed to achieve better coding performance. The bits of each CTU are assigned that the Lagrange multiplier $\lambda$ of a CTU is inversely proportional to its CTU level weight. This CTU level bit allocation scheme is applied to all the 360-degree video projection formats. Third, we propose a CTU row (CR) level rate control algorithm for the Equi-Rectangle Projection (ERP) format. Different CTUs in the same row in the ERP format are combined into a CR to provide more stable model parameters. The proposed algorithms are implemented in the newest video coding standard High Efficiency Video Coding (HEVC) reference software. The experimental results show that the proposed algorithm is able to achieve much better subjective and objective qualities as well as smaller bitrate errors compared with the state-of-the-art rate control algorithm.

[1]  Sanjit K. Mitra,et al.  A linear source model and a unified rate control algorithm for DCT video coding , 2002, IEEE Trans. Circuits Syst. Video Technol..

[2]  Yang Yang,et al.  A content-based rate control algorithm for screen content video coding , 2019, J. Vis. Commun. Image Represent..

[3]  Bin Li,et al.  QP refinement according to Lagrange multiplier for High Efficiency Video Coding , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[4]  Bin Li,et al.  $\lambda$-Domain Rate Control Algorithm for HEVC Scalable Extension , 2016, IEEE Transactions on Multimedia.

[5]  Lu Yu,et al.  Weighted-to-Spherically-Uniform Quality Evaluation for Omnidirectional Video , 2017, IEEE Signal Processing Letters.

[6]  Ju Liu,et al.  Coding modes-based frame skip avoidance scheme for low bit rate video coding , 2011, Journal of Real-Time Image Processing.

[7]  Wen Gao,et al.  Rate-GOP Based Rate Control for High Efficiency Video Coding , 2013, IEEE Journal of Selected Topics in Signal Processing.

[8]  Yu Zhou,et al.  SSIM-Based Game Theory Approach for Rate-Distortion Optimized Intra Frame CTU-Level Bit Allocation , 2016, IEEE Transactions on Multimedia.

[9]  Wen Gao,et al.  Rate-distortion analysis for H.264/AVC video coding and its application to rate control , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Yu Zhang,et al.  Optimized video coding for omnidirectional videos , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[11]  Sam Kwong,et al.  DCT Coefficient Distribution Modeling and Quality Dependency Analysis Based Frame-Level Bit Allocation for HEVC , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Zhu Li,et al.  Projection based advanced motion model for cubic mapping for 360-degree video , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[13]  Peng Wang,et al.  R-Lambda model based CTU-level rate control for intra frames in HEVC , 2017, Multimedia Tools and Applications.

[14]  Dapeng Wu,et al.  Real-Time Constant Objective Quality Video Coding Strategy in High Efficiency Video Coding , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Tiansong Li,et al.  The Bit Allocation Method Based on Inter-View Dependency for Multi-View Texture Video Coding , 2019, 2019 Data Compression Conference (DCC).

[16]  Yun Zhang,et al.  Rate Distortion Optimized Inter-View Frame Level Bit Allocation Method for MV-HEVC , 2015, IEEE Transactions on Multimedia.

[17]  Kenneth Rose,et al.  Rotational motion model for temporal prediction in 360 video coding , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[18]  Sam Kwong,et al.  Rate-Distortion Optimization of Rate Control for H.264 With Adaptive Initial Quantization Parameter Determination , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Sanjit K. Mitra,et al.  Low-delay rate control for DCT video coding via ?-domain source modeling , 2001, IEEE Trans. Circuits Syst. Video Technol..

[20]  Yiming Li,et al.  Spherical Domain Rate-Distortion Optimization for Omnidirectional Video Coding , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Bernd Girod,et al.  A Framework to Evaluate Omnidirectional Video Coding Schemes , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality.

[22]  Do-Kyoung Kwon,et al.  Rate Control for H.264 Video With Enhanced Rate and Distortion Models , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  荒木孝昌,et al.  Head-mounted display , 2015 .

[24]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[25]  Jianjun Lei,et al.  Region Adaptive R- $\lambda$ Model-Based Rate Control for Depth Maps Coding , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Yun Zhang,et al.  Interview Rate Distortion Analysis-Based Coarse to Fine Bit Allocation Algorithm for 3-D Video Coding , 2014, IEEE Transactions on Broadcasting.

[28]  Sanjit K. Mitra,et al.  . Optimum bit allocation and accurate rate control for video coding via ρ-domain source modeling , 2002, IEEE Trans. Circuits Syst. Video Technol..

[29]  Bin Li,et al.  A convolutional neural network-based approach to rate control in HEVC intra coding , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[30]  Shengxi Li,et al.  Optimal Bit Allocation at Frame Level for Rate Control in HEVC , 2019, IEEE Transactions on Broadcasting.

[31]  Tihao Chiang,et al.  A new rate control scheme using quadratic rate distortion model , 1997, IEEE Trans. Circuits Syst. Video Technol..

[32]  Zhenzhong Chen,et al.  An Optimized Rate Control for Low-Delay H.265/HEVC , 2019, IEEE Transactions on Image Processing.

[33]  Bin Li,et al.  $\lambda $ -Domain Optimal Bit Allocation Algorithm for High Efficiency Video Coding , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  King Ngi Ngan,et al.  An Efficient Frame-Content Based Intra Frame Rate Control for High Efficiency Video Coding , 2015, IEEE Signal Processing Letters.

[35]  Xiaoyan Sun,et al.  Optimal Bit Allocation for CTU Level Rate Control in HEVC , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Yiming Li,et al.  Spherical domain rate-distortion optimization for 360-degree video coding , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[37]  Ce Zhu,et al.  Inter-Block Dependency-Based CTU Level Rate Control for HEVC , 2020, IEEE Transactions on Broadcasting.

[38]  Li Li,et al.  Advanced Spherical Motion Model and Local Padding for 360° Video Compression , 2019, IEEE Transactions on Image Processing.

[39]  Sam Kwong,et al.  Data-Driven Rate Control for Rate-Distortion Optimization in HEVC Based on Simplified Effective Initial QP Learning , 2019, IEEE Transactions on Broadcasting.

[40]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Marie Babel,et al.  One-pass bitrate control for MPEG-4 Scalable Video Coding using ρ-domain , 2009, 2009 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting.

[42]  Chun Shi,et al.  A Novel Header Bits Estimation Scheme for H.264/AVC Standard , 2013 .

[43]  Chih-Wei Tang,et al.  Region-based rate control for 3D-HEVC based texture video coding , 2018, J. Vis. Commun. Image Represent..

[44]  Rong Xie,et al.  Weight-based bit allocation scheme for VR videos in HEVC , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[45]  Li Yang,et al.  Rate control schemes for panoramic video coding , 2018, J. Vis. Commun. Image Represent..