Video Encoder Architecture for Low-Delay Live-Streaming Events

Video-streaming events such as virtual classrooms and video conferences require a low delay between sender and receiver. In order to achieve this requirement, and to make full use of the bandwidth capacity of each receiver, each client can be provided with a personalized bitstream of which the bit rate is continuously adapted to his current network bandwidth capacity. However, such an approach requires an excessive amount of computationally complex video encoders. Therefore, this paper proposes an architecture based on coding information calculation (CIC) modules and residual encoder (RE) modules. The CIC modules calculate coding information for the video at certain bit rates whereas the RE modules use this information to skip all encoding steps of a traditional encoder, except for the encoding of the residual. By reducing the amount of bits used to encode the residual, the RE modules can then provide bitstreams with personalized bit rates for several users at the same time. Each CIC module has approximately the same computational complexity as a traditional encoder, whereas an RE module has the approximate complexity of a decoder. The proposed architecture was evaluated for the high efficiency video coding standard, showing that the system achieves its goal of drastically reducing the computational complexity of low-delay live-streaming with many participants and suggesting that using less than six CIC modules results in the best tradeoff between compression efficiency and computational complexity.

[1]  Byeungwoo Jeon,et al.  Early Skip Mode Decision for HEVC Encoder With Emphasis on Coding Quality , 2015, IEEE Transactions on Broadcasting.

[2]  Rik Van de Walle,et al.  Fast transcoding for video delivery by means of a control stream , 2012, 2012 19th IEEE International Conference on Image Processing.

[3]  Hongliang Li,et al.  A Fast HEVC Inter CU Selection Method Based on Pyramid Motion Divergence , 2014, IEEE Transactions on Multimedia.

[4]  Jaeho Lee,et al.  A Fast CU Size Decision Algorithm for HEVC , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Shih-Hsuan Yang,et al.  HEVC fast reference picture selection , 2015 .

[6]  Xinpeng Zhang,et al.  Fast TU size decision algorithm for HEVC encoders using Bayesian theorem detection , 2015, Signal Process. Image Commun..

[7]  Rob Lanphier Standardizing Real-Time Streaming Protocols , 1998, Computer.

[8]  Peter Lambert,et al.  Real-time complexity constrained encoding , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[9]  Zhi Liu,et al.  Adaptive Inter-Mode Decision for HEVC Jointly Utilizing Inter-Level and Spatiotemporal Correlations , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Yan Zhang,et al.  Fast inter-prediction mode decision algorithm for HEVC , 2015, ICSON.

[11]  Peter Lambert,et al.  Dynamic encoder profile optimisation for real-time video streaming applications , 2016 .

[12]  Michael J. Burke,et al.  Averaging Correlations: Expected Values and Bias in Combined Pearson rs and Fisher's z Transformations , 1998 .

[13]  Farzad Zargari,et al.  Two stage fast mode decision algorithm for intra prediction in HEVC , 2015, Multimedia Tools and Applications.

[14]  Kenneth Andersson,et al.  Guided just-in-time transcoding for cloud-based video platforms , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[15]  Yongdong Zhang,et al.  High Efficiency Video Coding: High Efficiency Video Coding , 2014 .

[16]  Bin Li,et al.  An Efficient Fast Mode Decision Method for Inter Prediction in HEVC , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Iraj Sodagar,et al.  The MPEG-DASH Standard for Multimedia Streaming Over the Internet , 2011, IEEE MultiMedia.

[18]  Munchurl Kim,et al.  A Novel Fast CU Encoding Scheme Based on Spatiotemporal Encoding Parameters for HEVC Inter Coding , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Magdy A. Bayoumi,et al.  Fast Motion Estimation System Using Dynamic Models for H.264/AVC Video Coding , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Kai-Kuang Ma,et al.  Fast Mode Decision for H.264/AVC Based on Macroblock Motion Activity , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Martin Reisslein,et al.  Efficient Multi-Rate Video Encoding for HEVC-Based Adaptive HTTP Streaming , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Zhi Liu,et al.  Effective CU Size Decision for HEVC Intracoding , 2014, IEEE Transactions on Image Processing.

[23]  José Luis Martínez,et al.  Fast intra mode decision algorithm based on texture orientation detection in HEVC , 2016, Signal Process. Image Commun..

[24]  Rik Van de Walle,et al.  Fast simultaneous video encoder for adaptive streaming , 2015, 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP).

[25]  NebutaFestival,et al.  Fast HEVC Encoding Decisions Using Data Mining , 2022 .

[26]  Jörn Ostermann,et al.  Fast Inter-Mode Decision in an H.264/AVC Encoder Using Mode and Lagrangian Cost Correlation , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Jar-Ferr Yang,et al.  A Low Complexity Detection of Discrete Cross Differences for Fast H.264/AVC Intra Prediction , 2008, IEEE Transactions on Multimedia.

[28]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[29]  Shang-Hong Lai,et al.  Fast H.264 Encoding Based on Statistical Learning , 2010, PCM.

[30]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[31]  Po-Chyi Su,et al.  Fast Intra-Coding for H.264/AVC by Using Projection-Based Predicted Block Residuals , 2013, IEEE Transactions on Multimedia.

[32]  Eduardo Peixoto,et al.  Inter-Prediction Optimizations for Video Coding Using Adaptive Coding Unit Visiting Order , 2016, IEEE Transactions on Multimedia.

[33]  Yo-Sung Ho,et al.  Categorization for fast intra prediction mode decision in H.264/AVC , 2010, IEEE Transactions on Consumer Electronics.

[34]  Rik Van de Walle,et al.  Efficient Bit Rate Transcoding for High Efficiency Video Coding , 2016, IEEE Transactions on Multimedia.

[35]  Jin Young Lee,et al.  A Fast Mode Decision Method Based on Motion Cost and Intra Prediction Cost for H.264/AVC , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Eckehard G. Steinbach,et al.  Block structure reuse for multi-rate high efficiency video coding , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[37]  Michael R. Frater,et al.  An Efficient Mode Selection Prior to the Actual Encoding for H.264/AVC Encoder , 2009, IEEE Transactions on Multimedia.

[38]  Yun He,et al.  Improvements on Fast Motion Estimation Strategy for H.264/AVC , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Byung-Gyu Kim,et al.  Fast Selective Intra-Mode Search Algorithm Based on Adaptive Thresholding Scheme for H.264/AVC Encoding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Yang Song,et al.  Motion Feature and Hadamard Coefficient-Based Fast Multiple Reference Frame Motion Estimation for H.264 , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Jae-Won Suh,et al.  A fast intra mode skip decision algorithm based on adaptive motion vector map , 2009, IEEE Transactions on Consumer Electronics.

[43]  Rae-Hong Park,et al.  Fast CU Partitioning Algorithm for HEVC Using an Online-Learning-Based Bayesian Decision Rule , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  Susanto Rahardja,et al.  Fast Coding Quad-Tree Decisions Using Prediction Residuals Statistics for High Efficiency Video Coding (HEVC) , 2016, IEEE Transactions on Broadcasting.

[45]  Jia-Ching Wang,et al.  Fast Mode Decision for H.264/AVC Based on Rate-Distortion Clustering , 2012, IEEE Transactions on Multimedia.