A rate distortion optimized transform for motion compensation residual

In this paper, we propose a rate distortion optimization based content adaptive transform method for motion compensation residuals. The proposed method utilizes pixel rearrangement to dynamically adjust the transform kernels to adapt to the residual content. Comparing with the traditional adaptive transforms, the highlight of this work is that it obtains the transform kernels from the decoded block, and hence it consumes only one overhead bit for each transform unit. Moreover, rate distortion optimization scheme is used to choose the best candidate kernels. Experimental results show that the proposed method achieves an average 0.35 dB gain of PSNR in comparison with the key technical areas (KTA) encoder.