Annealed learning based block transforms for HEVC video coding

Most of the recent video compression standards employ the Discrete Cosine Transform (DCT) for transforming the residual signal in order to remove spatial correlation and to achieve higher compression efficiency. However, by careful adaptation of transforms to the video content, a better set of integer transforms can be obtained. This paper proposes a new on-the-fly block-based transform optimization technique which involves first the classification of the residual blocks based on the cost of encoding the block, and then the generation of new optimized transforms for each class. An annealing based learning technique is further proposed in this paper in order to improve the performance of the optimization algorithm. The algorithm is tested using the latest HEVC test software where an optimized set of transforms is learned on the first frame of the HEVC test sequences and then applied to the subsequent frames in a Random Access (RA) and All Intra (AI) configuration. The results shows that this method can gain over 2% in terms of Bjontegaard Delta (BD)-rate compared to standard HEVC encoder in AI configuration and nearly 1.5% in RA.