Fast mode decision and early termination based on perceptual visual quality for HEVC encoders

This paper presents an early termination and fast mode decision for the HEVC encoder with a perceptual visual quality. For early termination of residual quad-tree, if the predicted errors for a transform block are not visible in terms of perceptual quality, the block is coded as the all-zero block and the transforms for the block and smaller blocks are skipped in RDO process. In addition, when a coding mode in the RDO-based mode decision stage is satisfactory in terms of visual quality, the coding mode is selected and the remaining coding modes requiring more bits are skipped. The proposed fast encoding algorithm is designed by pruning many coding modes when the coding errors with a coding mode having a small number of bits are not perceived by the human visual system. In addition, the proposed perceptual quality-based fast encoding algorithms can work with the existing objective quality-based fast decision algorithms. We found that the proposed algorithm can achieve 1.77 times acceleration of HM-16.8 with minimal perceptual visual degradation and objective quality distortion.