Quality of Experience (QoE)-Aware Fast Coding Unit Size Selection for HEVC Intra-Prediction

The exorbitant increase in the computational complexity of modern video coding standards, such as High Efficiency Video Coding (HEVC), is a compelling challenge for resource-constrained consumer electronic devices. For instance, the brute force evaluation of all possible combinations of available coding modes and quadtree-based coding structure in HEVC to determine the optimum set of coding parameters for a given content demand a substantial amount of computational and energy resources. Thus, the resource requirements for real time operation of HEVC has become a contributing factor towards the Quality of Experience (QoE) of the end users of emerging multimedia and future internet applications. In this context, this paper proposes a content-adaptive Coding Unit (CU) size selection algorithm for HEVC intra-prediction. The proposed algorithm builds content-specific weighted Support Vector Machine (SVM) models in real time during the encoding process, to provide an early estimate of CU size for a given content, avoiding the brute force evaluation of all possible coding mode combinations in HEVC. The experimental results demonstrate an average encoding time reduction of 52.38%, with an average Bjontegaard Delta Bit Rate (BDBR) increase of 1.19% compared to the HM16.1 reference encoder. Furthermore, the perceptual visual quality assessments conducted through Video Quality Metric (VQM) show minimal visual quality impact on the reconstructed videos of the proposed algorithm compared to state-of-the-art approaches.

[1]  David Flynn,et al.  HEVC Complexity and Implementation Analysis , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Zhan Ma,et al.  Early termination schemes for fast intra mode decision in High Efficiency Video Coding , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[3]  Yue Wang,et al.  Weighted support vector machine for data classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[4]  Jianjun Lei,et al.  Adaptive Fractional-Pixel Motion Estimation Skipped Algorithm for Efficient HEVC Motion Estimation , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[5]  Muhammad Usman Karim Khan,et al.  An adaptive complexity reduction scheme with fast prediction unit decision for HEVC intra encoding , 2013, 2013 IEEE International Conference on Image Processing.

[6]  Lu Yu,et al.  CU splitting early termination based on weighted SVM , 2013, EURASIP Journal on Image and Video Processing.

[7]  Jiawei Chen,et al.  Effective HEVC intra coding unit size decision based on online progressive Bayesian classification , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[8]  Warnakulasuriya Anil Chandana Fernando,et al.  Efficient coding unit size selection based on texture analysis for HEVC intra prediction , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[9]  Chun-Su Park Efficient intra-mode decision algorithm skipping unnecessary depth-modelling modes in 3D-HEVC , 2015 .

[10]  Munchurl Kim,et al.  Fast CU Splitting and Pruning for Suboptimal CU Partitioning in HEVC Intra Coding , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Krzysztof Okarma Adaptation of the Combined Image Similarity Index for Video Sequences , 2013, IP&C.

[12]  Wan-Chi Siu,et al.  Fast CU partition strategy for HEVC intra-frame coding using learning approach via random forests , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[13]  Lu Yu,et al.  Fast mode decision method for all intra spatial scalability in SHVC , 2014, 2014 IEEE Visual Communications and Image Processing Conference.

[14]  Biao Min,et al.  A Fast CU Size Decision Algorithm for the HEVC Intra Encoder , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Zhe Li,et al.  Gradient-based fast decision for intra prediction in HEVC , 2012, 2012 Visual Communications and Image Processing.

[16]  Zhan Ma,et al.  Fast Intra Mode Decision for High Efficiency Video Coding (HEVC) , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[18]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[19]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[20]  NebutaFestival,et al.  Fast HEVC Encoding Decisions Using Data Mining , 2022 .

[21]  Wan-Chi Siu,et al.  Novel Adaptive Algorithm for Intra Prediction With Compromised Modes Skipping and Signaling Processes in HEVC , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Gangyi Jiang,et al.  Effective Data Driven Coding Unit Size Decision Approaches for HEVC INTRA Coding , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[24]  Nan Hu,et al.  Fast Mode Selection for HEVC Intra-Frame Coding With Entropy Coding Refinement Based on a Transparent Composite Model , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Yuhui Zheng,et al.  Low complexity versatile video coding for traffic surveillance system , 2019 .

[26]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[27]  Long Xu,et al.  Machine Learning-Based Coding Unit Depth Decisions for Flexible Complexity Allocation in High Efficiency Video Coding , 2015, IEEE Transactions on Image Processing.

[28]  Jeong-Hoon Park,et al.  Block Partitioning Structure in the HEVC Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Zhenyu Liu,et al.  CU Partition Mode Decision for HEVC Hardwired Intra Encoder Using Convolution Neural Network , 2016, IEEE Transactions on Image Processing.

[30]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[31]  Anastasis A. Sofokleous,et al.  Review: H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia , 2005, Comput. J..

[32]  Gary J. Sullivan,et al.  Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC) , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Ping An,et al.  Fast CU size decision and mode decision algorithm for HEVC intra coding , 2013, IEEE Transactions on Consumer Electronics.

[34]  Tao Zhang,et al.  Fast Intra-Mode and CU Size Decision for HEVC , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  José Luis Martínez,et al.  Fast partitioning algorithm for HEVC Intra frame coding using machine learning , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[36]  Anil Fernando,et al.  Content-Adaptive Feature-Based CU Size Prediction for Fast Low-Delay Video Encoding in HEVC , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Benjamin Naumann,et al.  Learning And Soft Computing Support Vector Machines Neural Networks And Fuzzy Logic Models , 2016 .

[38]  Gangyi Jiang,et al.  Low Complexity HEVC INTRA Coding for High-Quality Mobile Video Communication , 2015, IEEE Transactions on Industrial Informatics.

[39]  Qiuwen Zhang,et al.  A Fast Intra Mode Decision Algorithm for HEVC Using Sobel Operator in Edge Detection , 2015, MUE 2015.

[40]  F. Bossen,et al.  Common test conditions and software reference configurations , 2010 .

[41]  Xuegong Zhang,et al.  Using class-center vectors to build support vector machines , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[42]  Yoonsik Choe,et al.  Fast Coding Unit size decision algorithm for intra coding in HEVC , 2013, 2013 IEEE International Conference on Consumer Electronics (ICCE).

[43]  Kemal Ugur,et al.  Intra Coding of the HEVC Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  Guifen Tian,et al.  Content adaptive prediction unit size decision algorithm for HEVC intra coding , 2012, 2012 Picture Coding Symposium.

[45]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[46]  Zhi Liu,et al.  Effective CU Size Decision for HEVC Intracoding , 2014, IEEE Transactions on Image Processing.

[47]  Dinesh Reddy,et al.  Fast coding unit partition search , 2012, 2012 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).