A joint optimization method of coding and transmission for conversational HD video service

Abstract Multipath transmission is considered to be a promising approach for supporting data delivery of conversational high-definition (HD) video service as it is able to aggregate more transmission resources, increase the reliability of data transmission, and thereby improve the user experience of service. However, most research achievements of multipath video transmission have mainly concentrated on the efficiency of video delivery while less attention has been paid to the dynamic video bit rate adjustment which is also important to improving the quality of such video service. To take full advantage of multipath video transmission, this paper presents a joint optimization method for conversational HD video service, taking into account the linkage between video coding and transmission. In the aspect of the video coding, a region rate based perceptual coding scheme is applied by the combination usage of the characteristics of such video service, which aims to adapt the output bit rate to match the dynamic network conditions. The scheme allocates different coding frequency or rate for different regions in a video frame according to their perceptual importance dynamically, and then skips some unimportant regions and encodes the key regions. Moreover, a multipath load distribution method is adopted to optimize the transmission process which schedules the coded frames over multiple paths by employing the capacity-limited water filling (CLWF) algorithm, to meet the stringent delay requirements of conversational HD video service. Furthermore, the transmission quality of each path is assessed and the coding parameters are adjusted periodically according to the data transmission status and feedback messages. Experiments are carried out with a simulation in OMNeT + + and the results demonstrate that the proposed method performs much better than the existing schemes in terms of data transmission and playback quality.

[1]  Hongke Zhang,et al.  CMT-QA: Quality-Aware Adaptive Concurrent Multipath Data Transfer in Heterogeneous Wireless Networks , 2013, IEEE Transactions on Mobile Computing.

[2]  Shengxi Li,et al.  Weight-based R-λ rate control for perceptual HEVC coding on conversational videos , 2015, Signal Process. Image Commun..

[3]  Chung-Ming Huang,et al.  Fast Retransmission for Concurrent Multipath Transfer (CMT) over Vehicular Networks , 2011, IEEE Communications Letters.

[4]  Zongliang Gan,et al.  Joint Spatial-Temporal Quality Improvement Scheme for H.264 Low Bit Rate Video Coding via Adaptive Frameskip , 2012, KSII Trans. Internet Inf. Syst..

[5]  Jun Huang,et al.  A novel scheduling approach to concurrent multipath transmission of high definition video in overlay networks , 2014, J. Netw. Comput. Appl..

[6]  Zhengguo Li,et al.  Region-of-Interest Based Resource Allocation for Conversational Video Communication of H.264/AVC , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Ming Wang,et al.  Content-Aware Concurrent Multipath Transfer for High-Definition Video Streaming over Heterogeneous Wireless Networks , 2016, IEEE Transactions on Parallel and Distributed Systems.

[8]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[9]  Weimin Lei,et al.  A general framework of multipath transport system based on application-level relay , 2014, Comput. Commun..

[10]  Chang-Su Kim,et al.  Efficient stereo video coding based on frame skipping for real-time mobile applications , 2008, IEEE Transactions on Consumer Electronics.

[11]  Yong Liu,et al.  Real-time bandwidth prediction and rate adaptation for video calls over cellular networks , 2016, MMSys.

[12]  Janardhan R. Iyengar,et al.  Concurrent multipath transfer using SCTP multihoming over independent end-to-end paths , 2006, TNET.

[13]  Weisi Lin,et al.  Rate control for videophone using local perceptual cues , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Myung Hoon Sunwoo,et al.  New Frame Rate Up-Conversion Algorithms With Low Computational Complexity , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Wen Gao,et al.  Rate-GOP Based Rate Control for High Efficiency Video Coding , 2013, IEEE Journal of Selected Topics in Signal Processing.

[16]  Keith Winstein,et al.  Salsify: Low-Latency Network Video through Tighter Integration between a Video Codec and a Transport Protocol , 2018, NSDI.

[17]  Bin Li,et al.  QP refinement according to Lagrange multiplier for High Efficiency Video Coding , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[18]  Azzedine Boukerche,et al.  A Multipath Video Streaming Solution for Vehicular Networks with Link Disjoint and Node-disjoint , 2015, IEEE Transactions on Parallel and Distributed Systems.

[19]  Shengxi Li,et al.  Region-of-Interest Based Conversational HEVC Coding with Hierarchical Perception Model of Face , 2014, IEEE Journal of Selected Topics in Signal Processing.

[20]  Antti Ylä-Jääski,et al.  Multipath Transmission for the Internet: A Survey , 2016, IEEE Communications Surveys & Tutorials.

[21]  Ishfaq Ahmad,et al.  A survey of rate control in HEVC and SHVC video encoding , 2017, 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[22]  Lei Yang,et al.  Considerations for application-layer multipath transport control , 2017, Int. J. Commun. Syst..

[23]  Luca De Cicco,et al.  A Google Congestion Control Algorithm for Real-Time Communication , 2012 .

[24]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Hongke Zhang,et al.  Cross-Layer Fairness-Driven Concurrent Multipath Video Delivery Over Heterogeneous Wireless Networks , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Touradj Ebrahimi,et al.  Semantic video analysis for adaptive content delivery and automatic description , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Chau Yuen,et al.  Distortion-Aware Concurrent Multipath Transfer for Mobile Video Streaming in Heterogeneous Wireless Networks , 2014, IEEE Transactions on Mobile Computing.

[28]  Eli Peli,et al.  Where people look when watching movies: Do all viewers look at the same place? , 2007, Comput. Biol. Medicine.

[29]  Xiaoyan Sun,et al.  Learning to Detect Video Saliency With HEVC Features , 2017, IEEE Transactions on Image Processing.

[30]  Jingyu Wang,et al.  OSIA: Out-of-order Scheduling for In-order Arriving in concurrent multi-path transfer , 2012, J. Netw. Comput. Appl..

[31]  Touradj Ebrahimi,et al.  Perceptual Video Compression: A Survey , 2012, IEEE Journal of Selected Topics in Signal Processing.

[32]  Bin Li,et al.  $\lambda $ -Domain Optimal Bit Allocation Algorithm for High Efficiency Video Coding , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Simon Lucey,et al.  Face alignment through subspace constrained mean-shifts , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[34]  Thomas Sikora,et al.  The MPEG-4 video standard verification model , 1997, IEEE Trans. Circuits Syst. Video Technol..

[35]  Nei Kato,et al.  Effective Delay-Controlled Load Distribution over Multipath Networks , 2011, IEEE Transactions on Parallel and Distributed Systems.

[36]  Eduardo Parente Ribeiro,et al.  Preventing quality degradation of video streaming using selective redundancy , 2016, Comput. Commun..

[37]  Kameswari Chebrolu,et al.  Bandwidth aggregation for real-time applications in heterogeneous wireless networks , 2006 .

[38]  Sangsu Jung,et al.  Multipath Video Real-Time Streaming by Field-Based Anycast Routing , 2014, IEEE Transactions on Multimedia.

[39]  Weimin Lei,et al.  An adaptive retransmission-based multipath transmission mechanism for conversational video , 2018, Int. J. Commun. Syst..

[40]  Christof Koch,et al.  Predicting human gaze using low-level saliency combined with face detection , 2007, NIPS.

[41]  Ming Wang,et al.  Energy-Minimized Multipath Video Transport to Mobile Devices in Heterogeneous Wireless Networks , 2016, IEEE Journal on Selected Areas in Communications.

[42]  Shaowei Liu,et al.  A Constrained Spatial-Temporal Frame Rate Control Model of Conversational Video for Multipath Transport System , 2018, 2018 3rd International Conference on Computer and Communication Systems (ICCCS).

[43]  Gregory W. Cermak,et al.  The Relationship Among Video Quality, Screen Resolution, and Bit Rate , 2011, IEEE Transactions on Broadcasting.

[44]  Minhua Zhou,et al.  An Overview of Tiles in HEVC , 2013, IEEE Journal of Selected Topics in Signal Processing.

[45]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[46]  Haoshan Shi,et al.  Dynamic Frame-Skipping Scheme for Live Video Encoders , 2010, 2010 International Conference on Multimedia Technology.

[47]  Shao-Yi Chien,et al.  Region-Based perceptual quality regulable bit allocation and rate control for video coding applications , 2012, 2012 Visual Communications and Image Processing.

[48]  Weimin Lei,et al.  CMT-SR: A selective retransmission based concurrent multipath transmission mechanism for conversational video , 2017, Comput. Networks.

[49]  David Zhang,et al.  A comprehensive evaluation of full reference image quality assessment algorithms , 2012, 2012 19th IEEE International Conference on Image Processing.

[50]  Houqiang Li,et al.  $\lambda $ Domain Rate Control Algorithm for High Efficiency Video Coding , 2014, IEEE Transactions on Image Processing.

[51]  Shaul Hochstein,et al.  At first sight: A high-level pop out effect for faces , 2005, Vision Research.

[52]  Jörg Ott,et al.  MPRTP: multipath considerations for real-time media , 2013, MMSys.

[53]  Sungjoo Yoo,et al.  Dual Motion Estimation for Frame Rate Up-Conversion , 2010, IEEE Transactions on Circuits and Systems for Video Technology.