Modeling and Optimization of High Frame Rate Video Transmission Over Wireless Networks

High frame rate (HFR) video is emerging as a new paradigm in popular multimedia applications (e.g., cloud gaming) to achieve smooth viewing experience perceived by end-users. In the context of HFR streaming video, end-to-end distortion and sending frame rate are equally important to the perceptual quality. This study presents a modeling-based approach to optimizing the HFR video transmission over wireless networks. First, we develop an analytical model dubbed FRIED (Frame Rate versus vIdEo Distortion) to characterize the tradeoff between sending frame rate and end-to-end video distortion. Second, we propose a Joint frAme Selection and FEC (Forward Error Correction) cOding (JASCO) approach based on the FRIED model to optimize the transmission performance. The efficacy of the proposed JASCO is evaluated through extensive semi-physical emulations in Exata involving H.264 video streaming. Experimental results show that JASCO outperforms the reference approaches in improving video peak signal-to-noise ratio (PSNR) at the same frame rate. Or conversely, JASCO is able to achieve higher received frame rate while guaranteeing the same video PSNR.

[1]  Martin Reisslein,et al.  Implications of Smoothing on Statistical Multiplexing of H.264/AVC and SVC Video Streams , 2009, IEEE Transactions on Broadcasting.

[2]  Jian He,et al.  iCloudAccess: Cost-Effective Streaming of Video Games From the Cloud With Low Latency , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Hayder Radha,et al.  Delay Constraint Error Control Protocol for Real-Time Video Communication , 2009, IEEE Transactions on Multimedia.

[4]  E. Gilbert.  Capacity of a burst-noise channel , 1960 .

[5]  Kajal T. Claypool,et al.  On frame rate and player performance in first person shooter games , 2007, Multimedia Systems.

[6]  Shipeng Li,et al.  High frame rate screen video coding for screen sharing applications , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[7]  Xiaodong Wang,et al.  Adaptive Transmission for Delay-Constrained Wireless Video , 2014, IEEE Transactions on Wireless Communications.

[8]  Jin Li,et al.  A hybrid FEC-ARQ protocol for low-delay lossless sequential data streaming , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[9]  Chau Yuen,et al.  Delay-Constrained High Definition Video Transmission in Heterogeneous Wireless Networks with Multi-Homed Terminals , 2016, IEEE Transactions on Mobile Computing.

[10]  Eryk Dutkiewicz,et al.  Cross-Layer Design for Proportional Delay Differentiation and Network Utility Maximization in Multi-Hop Wireless Networks , 2012, IEEE Transactions on Wireless Communications.

[11]  Jun Huang,et al.  Joint source-channel coding and optimization for mobile video streaming in heterogeneous wireless networks , 2013, EURASIP J. Wirel. Commun. Netw..

[12]  Mark Claypool,et al.  On the performance of OnLive thin client games , 2014, Multimedia Systems.

[13]  Pascal Frossard.  FEC performance in multimedia streaming , 2001, IEEE Communications Letters.

[14]  Pamela C. Cosman,et al.  End-to-end differentiation of congestion and wireless losses , 2003, TNET.

[15]  Ben Liang,et al.  Jointly optimal selection and scheduling for lossy transmission of dependent frames with delay constraint , 2014, 2014 IEEE 22nd International Symposium of Quality of Service (IWQoS).

[16]  Jiyan Wu,et al.  Robust bandwidth aggregation for real-time video delivery in integrated heterogeneous wireless networks , 2015, Multimedia Tools and Applications.

[17]  Chau Yuen,et al.  A low-latency scheduling approach for high-definition video streaming in a heterogeneous wireless network with multihomed clients , 2014, Multimedia Systems.

[18]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[19]  Athanasios V. Vasilakos,et al.  Availability-Aware Multimedia Scheduling in Heterogeneous Wireless Networks , 2011, IEEE Transactions on Vehicular Technology.

[20]  Bernd Girod,et al.  What's wrong with mean-squared error? , 1993 .

[21]  Chau Yuen,et al.  Trading Delay for Distortion in One-Way Video Communication Over the Internet , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Chun-Ying Huang,et al.  Measuring the latency of cloud gaming systems , 2011, ACM Multimedia.

[23]  Jiyan Wu,et al.  A low latency scheduling approach for high definition video streaming over heterogeneous wireless networks , 2013, 2013 IEEE Global Communications Conference (GLOBECOM).

[24]  Chau Yuen,et al.  Distortion-Aware Concurrent Multipath Transfer for Mobile Video Streaming in Heterogeneous Wireless Networks , 2014, IEEE Transactions on Mobile Computing.

[25]  Chau Yuen,et al.  Enabling Adaptive High-Frame-Rate Video Streaming in Mobile Cloud Gaming Applications , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Chau Yuen,et al.  Goodput-Aware Load Distribution for Real-Time Traffic over Multipath Networks , 2015, IEEE Transactions on Parallel and Distributed Systems.

[27]  Zhan Ma,et al.  Modeling the impact of frame rate on perceptual quality of video , 2008, 2008 15th IEEE International Conference on Image Processing.

[28]  Gerhard Haßlinger,et al.  2-State (semi-)Markov processes beyond Gilbert-Elliott: Traffic and channel models based on 2nd order statistics , 2013, 2013 Proceedings IEEE INFOCOM.

[29]  Alan C. Bovik,et al.  Automatic Prediction of Perceptual Image and Video Quality , 2013, Proceedings of the IEEE.

[30]  Gabriella Olmo,et al.  Slice Sorting for Unequal Loss Protection of Video Streams , 2008, IEEE Signal Processing Letters.

[31]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[32]  Koushik Kar,et al.  A Transport Protocol to Exploit Multipath Diversity in Wireless Networks , 2012, IEEE/ACM Transactions on Networking.

[33]  Cheng-Hsin Hsu,et al.  On the Quality of Service of Cloud Gaming Systems , 2014, IEEE Transactions on Multimedia.

[34]  Kensuke Fukuda,et al.  Seven Years and One Day: Sketching the Evolution of Internet Traffic , 2009, IEEE INFOCOM 2009.

[35]  Chau Yuen,et al.  Leveraging the Delay-Friendliness of TCP With FEC Coding in Real-Time Video Communication , 2015, IEEE Transactions on Communications.

[36]  Wei Song,et al.  Performance Analysis of Probabilistic Multipath Transmission of Video Streaming Traffic over Multi-Radio Wireless Devices , 2012, IEEE Transactions on Wireless Communications.

[37]  Sampath Rangarajan,et al.  MuVi: a multicast video delivery scheme for 4g cellular networks , 2012, Mobicom '12.

[38]  Wei Tu,et al.  Distributed scheduling scheme for video streaming over multi-channel multi-radio multi-hop wireless networks , 2010, IEEE Journal on Selected Areas in Communications.

[39]  Tansu Alpcan,et al.  Distributed Rate Allocation Policies for Multihomed Video Streaming Over Heterogeneous Access Networks , 2009, IEEE Transactions on Multimedia.

[40]  Yue Deng,et al.  Joint non-Gaussian denoising and superresolving of raw high frame rate videos. , 2014, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[41]  Kameswari Chebrolu,et al.  Selective frame discard for interactive video , 2004, 2004 IEEE International Conference on Communications (IEEE Cat. No.04CH37577).

[42]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[43]  Bernd Girod,et al.  Analysis of video transmission over lossy channels , 2000, IEEE Journal on Selected Areas in Communications.

[44]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[45]  Yao Zhao,et al.  Dynamic Sub-GOP Forward Error Correction Code for Real-Time Video Applications , 2012, IEEE Transactions on Multimedia.

[46]  Hayder Radha,et al.  Hybrid Erasure-Error Protocols for Wireless Video , 2007, IEEE Transactions on Multimedia.

[47]  Weihua Zhuang,et al.  Energy and Content Aware Multi-Homing Video Transmission in Heterogeneous Networks , 2013, IEEE Transactions on Wireless Communications.