AN efficient algorithm for joint QP and quantization optimization for H.264/AVC

We describe an efficient algorithm for jointly optimizing the quantization parameter (QP) and the quantization decisions in H.264/AVC. Based on bitrate estimation for quantized and entropy coded coefficients in H.264/AVC, a technique was introduced to select only a small subset of transform coefficients which have the most significant impact on the overall quantization performance. Only the quantization decisions for the coefficients in this subset need to be explicitly examined along with the QP. Simulation results showed that the proposed algorithm achieves similar quantization performances as existing state-of-the-art optimized algorithms at roughly half the complexity.