Towards A Live Software Decoder Implementation For The Upcoming Versatile Video Coding (VVC) Codec

Versatile Video Coding (VVC) is the emerging video coding standard to be finalized by the Joint Video Experts Team in July 2020. Compared to its predecessor, the High Efficiency Video Coding (HEVC) standard, VVC provides 50% bit-rate reduction at comparable visual quality for natural video content in high-definition (HD) and ultra high-definition (UHD) resolution. To achieve this, the standard incorporates more advanced and generalized algorithms, leading to an increase in computational complexity. This includes for example additional in-loop filters, decoder-side motion refinement and search as well as an increased number of transforms, which creates a grand challenge for implementers to achieve live decoding on general-purpose CPUs. In this paper, the work on an efficient software decoder implementation for the upcoming VVC standard is described, including optimization of sample operations using single instruction multiple data (SIMD) instructions and parallelization approaches with multithreading. As a result, the presented decoder can perform live decoding of 10bit HD video at 60 frames per second (fps) and 10bit UHD video at 30fps on modern mobile consumer hardware, showcasing that VVC live decoding is possible already right before finalization of the standard.

[1]  Detlev Marpe,et al.  HEVC performance and complexity for 4K video , 2013, 2013 IEEE Third International Conference on Consumer Electronics ¿ Berlin (ICCE-Berlin).

[2]  Olivier Déforges,et al.  Performance and Computational Complexity of the Future Video Coding , 2018, 2018 IEEE International Workshop on Signal Processing Systems (SiPS).

[3]  Michel Kieffer,et al.  Low Complexity Versatile Video Coding (VVC) for Low Bitrate Applications , 2019, 2019 8th European Workshop on Visual Information Processing (EUVIP).

[4]  Ben H. H. Juurlink,et al.  Parallel video decoding in the emerging HEVC standard , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Ruby B. Lee.  Accelerating multimedia with enhanced microprocessors , 1995, IEEE Micro.

[6]  David Flynn,et al.  HEVC Complexity and Implementation Analysis , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Eckehard Steinbach,et al.  Low Complexity Decoder Side Motion Vector Refinement for VVC , 2019, 2019 Picture Coding Symposium (PCS).

[8]  Gary J. Sullivan,et al.  General Video Coding Technology in Responses to the Joint Call for Proposals on Video Compression With Capability Beyond HEVC , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Heiko Schwarz,et al.  Video Compression Using Generalized Binary Partitioning, Trellis Coded Quantization, Perceptually Optimized Encoding, and Advanced Prediction and Transform Coding , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Ben H. H. Juurlink,et al.  SIMD Acceleration for HEVC Decoding , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Gary J. Sullivan,et al.  The Joint Exploration Model (JEM) for Video Compression With Capability Beyond HEVC , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Detlev Marpe,et al.  Video compression using context-based adaptive arithmetic coding , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[14]  W. Hamidouche,et al.  Complexity Reduction Opportunities in the Future VVC Intra Encoder , 2019, 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP).