Rate-Distortion Optimization for Video Coding under Given Computational Complexity

Rate-distortion optimization (RDO) is widely applied in video coding, which aims at minimizing the coding distortion under a target coding rate. Conventionally, RDO in video coding does not take into account the coding complexity. However, because of the diversity of video applications, the video encoders in different applications may have different requirements of or limitation on the computational complexity. Therefore, it is desirable for video encoders to perform RDO in flexible computational complexity. In this paper, we propose a novel RDO scheme under the given computational complexity for the latest H.265/HEVC standard. A model for prediction of the rate-distortion cost (RD cost) is first established based on a pre-searching process. Then according to the predicted RD cost, the rate-distortion-complexity (R-D-C) characteristics of different coding tree units (CTUs) are analyzed. Finally, the total complexity budget is properly allocated to different CTUs according to their R-D-C characteristics. Experimental results demonstrate that, compared with x265, the proposed algorithm can reduce, on average, the BD-rate by 18.8% under the same requirements of encoding speed.