Complexity Estimation for Load Balancing of 360-Degree Intra Versatile Video Coding

The ever increasing demand for image and video content poses new requirements to support higher resolutions and richer representation formats, creating new challenges in coding algorithms. The forthcoming Versatile Video Coding (VVC) standard aims to increase the coding efficiency of existing algorithms and it is particularly suitable for Ultra-High Definition (UHD) resolutions and 360° video. However, since coding efficiency gains are obtained at the cost of increased complexity, fast computational approaches are needed to cope with realtime requirements, such as parallel processing. Thus, this work presents a contribution towards efficient parallel encoding of 360° video, based on coding complexity estimation and nonuniform data-level splitting (slice-based) for load balancing across multiple processors. A machine learning approach is proposed to estimate the complexity of intra coding VVC, using uncorrelated features, obtained through Principal Component Analysis (PCA) and Extremely Randomised Trees (ERT). Then, a complexity-balanced slice partition is devised, taking advantage of the clustered complexity inherent to Equirectangular Projection (ERP). It is shown that coding complexity is estimated with an accuracy of 92.25%, and the encoding time is reduced by 8.50%, when compared to the case where the 360° frames are evenly split.

[1]  Heiko Schwarz,et al.  Fast Partitioning Decision Strategies for The Upcoming Versatile Video Coding (VVC) Standard , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[2]  Chao Chen,et al.  Using Random Forest to Learn Imbalanced Data , 2004 .

[3]  D. Woolley.  The White Paper. , 1972, British medical journal.

[4]  Thanasis Loukopoulos,et al.  A Fast Heuristic for Tile Partitioning and Processor Assignment in Hevc , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[5]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[6]  Stefan Winkler,et al.  Image complexity and spatial information , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[7]  Stéphane Coulombe,et al.  Fast HEVC Intra Mode Decision Based on RDO Cost Prediction , 2019, IEEE Transactions on Broadcasting.

[8]  Miska M. Hannuksela,et al.  Standardization status of 360 degree video coding and delivery , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[9]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[10]  Mathias Wien,et al.  Guest Editorial Immersive Video Coding and Transmission , 2019, IEEE J. Emerg. Sel. Topics Circuits Syst..

[11]  Thanasis Loukopoulos,et al.  Slice-based parallelization in HEVC encoding: Realizing the potential through efficient load balancing , 2016, 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP).

[12]  Minhua Zhou,et al.  An Overview of Tiles in HEVC , 2013, IEEE Journal of Selected Topics in Signal Processing.

[13]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[14]  Mohamed-Chaker Larabi,et al.  A Low-Complexity Video Encoder for Equirectangular Projected 360 Video Content , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Daniel Ménard,et al.  Random Forest Oriented Fast QTBT Frame Partitioning , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Yong-Jo Ahn,et al.  Complexity model based load-balancing algorithm for parallel tools of HEVC , 2013, 2013 Visual Communications and Image Processing (VCIP).

[17]  Thanasis Loukopoulos,et al.  Adaptive Tile Parallelization for Fast Video Encoding in HEVC , 2016, 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[18]  John P. Snyder,et al.  Map Projections: A Working Manual , 2012 .