High-speed implementation of rate-distortion optimized quantization for H.264/AVC

Rate-distortion optimized quantization (RDOQ) is generally employed in video coding as an effective tool for pursuing high coding efficiency. Nevertheless, RDOQ requires an exhaustive search over multiple candidates to determine the optimal quantized level by comparing their rate-distortion cost, which leads to considerable complexity in practice. The utilization of inherently strong data dependency and highly sequential process in entropy coding to obtain the information of the bit-rate for each candidate further aggravates the issue of large computational burden. In this paper, a high-speed implementation of RDOQ is proposed to substitute the conventional RDOQ in H.264/AVC. The proposed scheme is low in computational complexity and economical in the requirement of hardware resources. The candidates of quantized levels that are unlikely to be selected are firstly excluded, and then a bit-rate estimation model is constructed for the rest candidates, to avoid the actual entropy coding of context-adaptive binary arithmetic coding for the calculation of the rate-distortion cost. Eventually, the fast algorithm of RDOQ is designed to determine the optimal quantized level, which supports parallel processing and is therefore favorable for hardware implementations. Experimental results demonstrate that the proposed method can decrease the time of RDOQ by 60.5 % in average with a superior RD performance in terms of encoding.