Low complexity high efficiency coding of light fields using ensemble classifiers

Abstract Light field images can be efficiently compressed using standard video codecs, such as the High Efficiency Video Coding (HEVC). However, the huge amount of data combined with the high computational complexity of HEVC, poses limitations on high-speed light field capturing and storage. This paper presents a contribution for low complexity encoding of light fields, in different formats using HEVC, based on a Random Forests ensemble classifier. Optimal features for training the classifier are found through a score fusion based approach. Using the HEVC still image profile, the proposed method gives speed-up of 56.23% for sub-aperture images. For pseudo video format, the proposed method outperforms others available in the literature, yielding an average speed-up of 62.18%, 56.54% and 44.73% for Random Access, Low-delay Main and All-Intra profiles respectively, with negligible decrease in RD performance. These are novel results in fast coding of light fields, which are useful for further research and benchmarking.

[1]  Jie Chen,et al.  Light Field Image Compression Based on Bi-Level View Compensation With Rate-Distortion Optimization , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Long Xu,et al.  Machine Learning-Based Coding Unit Depth Decisions for Flexible Complexity Allocation in High Efficiency Video Coding , 2015, IEEE Transactions on Image Processing.

[3]  Pao-Chi Chang,et al.  Rough mode cost-based fast intra coding for high-efficiency video coding , 2017, J. Vis. Commun. Image Represent..

[4]  Touradj Ebrahimi,et al.  Comparison and Evaluation of Light Field Image Coding Approaches , 2017, IEEE Journal of Selected Topics in Signal Processing.

[5]  M. Levoy,et al.  The light field , 1939 .

[6]  David Flynn,et al.  HEVC Complexity and Implementation Analysis , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Yan Zhang,et al.  Fast inter-prediction mode decision algorithm for HEVC , 2015, ICSON.

[8]  Lu Yu,et al.  CU splitting early termination based on weighted SVM , 2013, EURASIP Journal on Image and Video Processing.

[9]  Li Li,et al.  Pseudo-sequence-based light field image compression , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[10]  Soon Myoung Chung,et al.  Combining Multiple Feature Selection Methods for Text Categorization by Using Rank-Score Characteristics , 2009, 2009 21st IEEE International Conference on Tools with Artificial Intelligence.

[11]  Nicoletta Dessì,et al.  An evolutionary method for combining different feature selection criteria in microarray data classification , 2009 .

[12]  Cristian Perra,et al.  High efficiency coding of light field images based on tiling and pseudo-temporal data arrangement , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[13]  Pedro A. Amado Assunção,et al.  Lossless coding of light field images based on minimum-rate predictors , 2018, J. Vis. Commun. Image Represent..

[14]  Kalyan Goswami,et al.  A Design of Fast High-Efficiency Video Coding Scheme Based on Markov Chain Monte Carlo Model and Bayesian Classifier , 2018, IEEE Transactions on Industrial Electronics.

[15]  Qionghai Dai,et al.  Light Field Image Processing: An Overview , 2017, IEEE Journal of Selected Topics in Signal Processing.

[16]  Yun Li,et al.  Coding of Focused Plenoptic Contents by Displacement Intra Prediction , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[18]  Touradj Ebrahimi,et al.  JPEG Pleno: Toward an Efficient Representation of Visual Reality , 2016, IEEE MultiMedia.

[19]  Kebin Jia,et al.  Fast Intra CTU Depth Decision for HEVC , 2018, IEEE Access.

[20]  Román Hermida,et al.  Complexity reduction in the HEVC/H265 standard based on smooth region classification , 2018, Digit. Signal Process..

[21]  Guilherme Corrêa,et al.  Performance and Computational Complexity Assessment of High-Efficiency Video Encoders , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Dan Lelescu,et al.  Representation and coding of light field data , 2004, Graph. Model..

[23]  Sergio Bampi,et al.  Fast Coding Unit Partition Decision for HEVC Using Support Vector Machines , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Wan-Chi Siu,et al.  Fast CU partition strategy for HEVC intra-frame coding using learning approach via random forests , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[25]  Xinpeng Zhang,et al.  An Effective CU Size Decision Method for HEVC Encoders , 2013, IEEE Transactions on Multimedia.

[26]  Gangyi Jiang,et al.  Machine learning based fast H.264/AVC to HEVC transcoding exploiting block partition similarity , 2016, J. Vis. Commun. Image Represent..

[27]  Jian Wang,et al.  Exploiting Data Mining for Fast Inter Prediction Mode Decision in HEVC , 2018, Mob. Networks Appl..

[28]  Sam Kwong,et al.  Spatial/temporal motion consistency based MERGE mode early decision for HEVC , 2017, Journal of Visual Communication and Image Representation.