Armor: adjusting repair and media scaling with operations research for streaming video

Streaming multimedia quality is impacted by two main factors: capacity constraint and packet loss. To match the capacity constraint while preserving real-time playout, media scaling can be used to discard the encoded multimedia content that has the least impact on perceived video quality. To limit the impact of lost packets, repair techniques; e.g. forward error correction (FEC), can be used to repair frames damaged by packet loss. However, adding data to facilitate repair requires further reduction of the original multimedia data, making the decision of how much repair data to use of critical importance. Assuming a limited network capacity and the availability of an estimate of the current packet loss rate along a flow path, selecting the best distribution of FEC packets for video frames with inherent interframe encoding dependencies can be cast as a constraint optimization problem that attempts to optimize the quality of the video stream. This thesis presents an Adjusting Repair and Media scaling with Operations Research (ARMOR) system. An analytical model is derived for streaming video with FEC and media scaling. Given parameters to represent network loss as well as video frame types and sizes, if the number of FEC packets per video frame type and media scaling pattern is specified, the model can estimate the video quality at the receiver side. The model is then used in an operations research algorithm to adjust the FEC strength and media scaling level to yield the best quality under the capacity constraint. Four different combinations of FEC type and media scaling method are studied: Media Independent FEC with Temporal Scaling (MITS), Media Independent FEC with Quality Scaling (MIQS), Media Independent FEC with Temporal and Quality Scaling (MITQS), and Media Dependent FEC with Quality Scaling (MDQS). The analytical experiments show: (1) adjusting FEC always achieves a higher video quality than streaming video without FEC or with a fixed amount of FEC; (2) Quality Scaling usually works better than Temporal Scaling; and (3) Media Dependent FEC (MDFEC) is typically less effective than Media Independent FEC (MIFEC). A user study is presented with results from 74 participants analysis shows that the ARMOR model can accurately estimate users' perceptual quality. Well-designed simulations and a realistic system implementation suggests the ARMOR system can practically improve the quality of streaming video.

[1]  Mark W. Garrett,et al.  Modeling and generation of self-similar vbr video traffic , 1994, SIGCOMM 1994.

[2]  Deborah Estrin,et al.  RAP: An end-to-end rate-based congestion control mechanism for realtime streams in the Internet , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[3]  Ketan Mayer-Patel,et al.  An MPEG performance model and its application to adaptive forward error correction , 2002, MULTIMEDIA '02.

[4]  Wu-chi Feng,et al.  Quality-adaptive media streaming by priority drop , 2003, NOSSDAV '03.

[5]  Oliver Rose,et al.  Statistical properties of MPEG video traffic and their impact on traffic modeling in ATM systems , 1995, Proceedings of 20th Conference on Local Computer Networks.

[6]  Thierry Turletti,et al.  Adaptive error control for packet video in the Internet , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[7]  Nichols,et al.  Measurement of Windows Streaming Media , 2004 .

[8]  V. Hardman,et al.  A survey of packet loss recovery techniques for streaming audio , 1998, IEEE Network.

[9]  John W. Woods,et al.  Motion compensated multiresolution transmission of high definition video , 1994, IEEE Trans. Circuits Syst. Video Technol..

[10]  Marwan Krunz,et al.  Statistical characteristics and multiplexing of MPEG streams , 1995, Proceedings of INFOCOM'95.

[11]  Jill M. Boyce,et al.  Packet loss effects on MPEG video sent over the public Internet , 1998, MULTIMEDIA '98.

[12]  Vern Paxson,et al.  End-to-end Internet packet dynamics , 1997, SIGCOMM '97.

[13]  Didier Le Gall,et al.  MPEG: a video compression standard for multimedia applications , 1991, CACM.

[14]  Donald F. Towsley,et al.  Inferring TCP connection characteristics through passive measurements , 2004, IEEE INFOCOM 2004.

[15]  Mark Claypool,et al.  Adjusting forward error correction with temporal scaling for TCP-friendly streaming MPEG , 2005, TOMCCAP.

[16]  Sally Floyd,et al.  Promoting the use of end-to-end congestion control in the Internet , 1999, TNET.

[17]  Ulrich Benzler.  Spatial scalable video coding using a combined subband-DCT approach , 2000, IEEE Trans. Circuits Syst. Video Technol..

[18]  B. Barden.  Recommendations on queue management and congestion avoidance in the Internet , 1998 .

[19]  Georg Carle,et al.  Survey of error recovery techniques for IP-based audio-visual multicast applications , 1997, IEEE Netw..

[20]  Injong Rhee,et al.  Error control techniques for interactive low-bit rate video transmission over the Internet , 1998, SIGCOMM '98.

[21]  Avideh Zakhor,et al.  Real-Time Internet Video Using Error Resilient Scalable Compression and TCP-Friendly Transport Protocol , 1999, IEEE Trans. Multim..

[22]  Sungrae Cho,et al.  Adaptive error control for hybrid (satellite-terrestrial) networks , 1999, WCNC. 1999 IEEE Wireless Communications and Networking Conference (Cat. No.99TH8466).

[23]  Marek Domanski,et al.  Spatio-temporal scalability using modified MPEG-2 predictive video coding , 2000, 2000 10th European Signal Processing Conference.

[24]  M. Angela Sasse,et al.  Sharp or smooth?: comparing the effects of quantization vs. frame rate for streamed video , 2004, CHI '04.

[25]  Mohammed Ghanbari,et al.  Standard Codecs: Image Compression to Advanced Video Coding , 2003 .

[26]  Mark Claypool,et al.  A model for MPEG with forward error correction and TCP-friendly bandwidth , 2003, NOSSDAV '03.

[27]  Pascal Frossard,et al.  Joint source/FEC rate selection for quality-optimal MPEG-2 video delivery , 2001, IEEE Trans. Image Process..

[28]  Mark Handley,et al.  Reliable Audio for Use over the Internet , 2006 .

[29]  Mark Claypool,et al.  Guidelines for Selecting Practical MPEG Group of Pictures , 2006, EuroIMSA.

[30]  A. Rix.  Advances in objective quality assessment of speech over analogue and packet based networks , 1999 .

[31]  Madhu Sudan,et al.  Priority encoding transmission , 1996, IEEE Trans. Inf. Theory.

[32]  Alfred C. Weaver,et al.  Adaptive Error Control for Multimedia Data Transfers , 1992, [1992] Proceedings International Workshop on Advanced Communications and Applications for High Speed Networks.

[33]  Andrew T. Campbell,et al.  Utility-based Network Adaptation for MPEG-4 Systems , 1999 .

[34]  Harsha Sirisena,et al.  Hybrid error control mechanism for video transmission in the wireless IP networks , 1999, 10th IEEE Workshop on Local and Metropolitan Area Networks. Selected Papers (IEEE Cat. No.99EX512).

[35]  Huahui Wu.  A Study of Video Motion and Scene Complexity , 2006 .

[36]  Dmitri Loguinov,et al.  Impact of FEC overhead on scalable video streaming , 2005, NOSSDAV '05.

[37]  Wei Wang,et al.  QoS-sensitive transport of real-time MPEG video using adaptive forward error correction , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[38]  Deborah Estrin,et al.  Quality adaptation for congestion controlled video playback over the Internet , 1999, SIGCOMM '99.

[39]  Bernd Girod,et al.  Error-resilient coding for H.263 , 1999 .

[40]  Weiping Li,et al.  Overview of fine granularity scalability in MPEG-4 video standard , 2001, IEEE Trans. Circuits Syst. Video Technol..

[41]  Barry G. Haskell,et al.  I/P/B frame type decision by collinearity of displacements , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[42]  Sheila S. Hemami,et al.  A comparison of temporal scalability techniques , 1999, IEEE Trans. Circuits Syst. Video Technol..

[43]  Wu-chi Feng,et al.  Under the Plastic: A Quantitative Look at DVD Video Encoding and Its Impact on Video Modeling , 2003 .

[44]  Nick Feamster,et al.  Packet Loss Recovery for Streaming Video , 2002 .

[45]  Andrew T. Campbell,et al.  A Utility-Based Approach for Quantitative Adaptation in Wireless Packet Networks , 2001, Wirel. Networks.

[46]  Hayder Radha,et al.  Interleaved Source Coding ( ISC ) for Predictive Video over ERASURE-Channels , 2004 .

[47]  Nick Feamster,et al.  On the Interactions Between Layered Quality Adaptation and Congestion Control for Streaming Video , 2002 .

[48]  Donald F. Towsley,et al.  Adaptive FEC-based error control for Internet telephony , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[49]  Mark Claypool,et al.  Video redundancy: a best-effort solution to network data loss , 1999, MULTIMEDIA '99.

[50]  Joan L. Mitchell,et al.  MPEG Video Compression Standard , 1996, Springer US.

[51]  A. Mena,et al.  An empirical study of real audio traffic , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[52]  Margaret H. Pinson,et al.  On the impact of policing and rate guarantees in DiffServ networks: a video streaming application perspective , 2001, SIGCOMM 2001.

[53]  Mark Claypool,et al.  Adjusting forward error correction with quality scaling for streaming MPEG , 2005, NOSSDAV '05.

[54]  Mark Claypool,et al.  Improving Multimedia Streaming with Content-Aware Video Scaling , 2002, JCIS.

[55]  Yali Zhu,et al.  MEASUREMENT OF THE CONGESTION RESPONSIVENESS OF REALPLAYER STREAMING VIDEO OVER UDP , 2003 .

[56]  Hiroyuki Katata,et al.  Temporal-scalable coding based on image content , 1997, IEEE Trans. Circuits Syst. Video Technol..

[57]  Mark Handley,et al.  Equation-based congestion control for unicast applications , 2000, SIGCOMM.

[58]  James X. Qiu,et al.  Error Control for Integrated Wireless and Wireline Networks , 1998 .

[59]  John G. Beerends,et al.  A Perceptual Audio Quality Measure Based on a Psychoacoustic Sound Representation , 1992 .

[60]  Luigi Rizzo,et al.  Effective erasure codes for reliable computer communication protocols , 1997, CCRV.

[61]  Mark Claypool,et al.  Using redundancy to repair video damaged by network data loss , 1999, Electronic Imaging.

[62]  Walter Willinger,et al.  Analysis, modeling and generation of self-similar VBR video traffic , 1994, SIGCOMM.

[63]  Stefan Winkler,et al.  Audiovisual quality evaluation of low-bitrate video , 2005, IS&T/SPIE Electronic Imaging.

[64]  Mark Carson,et al.  NIST Net: a Linux-based network emulation tool , 2003, CCRV.

[65]  Sally Floyd,et al.  TCP and explicit congestion notification , 1994, CCRV.

[66]  Naoki Wakamiya,et al.  MPEG-TFRCP: video transfer with TCP-friendly rate control protocol , 2001, ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No.01CH37240).

[67]  Kenneth J. Christensen,et al.  A new adaptive FEC loss control algorithm for voice over IP applications , 2000, Conference Proceedings of the 2000 IEEE International Performance, Computing, and Communications Conference (Cat. No.00CH37086).

[68]  Thinh Nguyen,et al.  Distributed Video Streaming with Forward Error Correction , 2003 .

[69]  Mark Claypool,et al.  An empirical study of realvideo performance across the internet , 2001, IMW '01.

[70]  Nariman Farvardin,et al.  Three-dimensional subband coding of video , 1995, IEEE Trans. Image Process..

[71]  Stefano Tubaro,et al.  A novel error-concealment algorithm for an unbalanced multiple description coding architecture , 2004 .

[72]  Ratul Mahajan,et al.  Controlling high-bandwidth flows at the congested router , 2001, Proceedings Ninth International Conference on Network Protocols. ICNP 2001.

[73]  Hayder Radha,et al.  Measurement study of low-bitrate internet video streaming , 2001, IMW '01.

[74]  Avideh Zakhor,et al.  Multirate 3-D subband coding of video , 1994, IEEE Trans. Image Process..

[75]  Donald F. Towsley,et al.  Modeling TCP throughput: a simple model and its empirical validation , 1998, SIGCOMM '98.

[76]  Yali Zhu,et al.  Using interleaving to ameliorate the effects of packet loss in a video stream , 2003, 23rd International Conference on Distributed Computing Systems Workshops, 2003. Proceedings..

[77]  Yutaka Yokoyama.  Adaptive GOP structure selection for real-time MPEG-2 video encoding , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[78]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[79]  Srinivasan Seshan,et al.  An integrated congestion management architecture for Internet hosts , 1999, SIGCOMM '99.