Rate-distortion optimized streaming of packetized media

This paper addresses the problem of streaming packetized media over a lossy packet network in a rate-distortion optimized way. We show that although the data units in a media presentation generally depend on each other according to a directed acyclic graph, the problem of rate-distortion optimized streaming of an entire presentation can be reduced to the problem of error-cost optimized transmission of an isolated data unit. We show how to solve the latter problem in a variety of scenarios, including the important common scenario of sender-driven streaming with feedback over a best-effort network, which we couch in the framework of Markov decision processes. We derive a fast practical algorithm for nearly optimal streaming in this scenario, and we derive a general purpose iterative descent algorithm for locally optimal streaming in arbitrary scenarios. Experimental results show that systems based on our algorithms have steady-state gains of 2-6 dB or more over systems that are not rate-distortion optimized. Furthermore, our systems essentially achieve the best possible performance: the operational distortion-rate function of the source at the capacity of the packet erasure channel.

[1]  Sorina Dumitrescu,et al.  Globally optimal uneven error-protected packetization of scalable code streams , 2004, IEEE Trans. Multim..

[2]  Antonio Ortega,et al.  Expected Run-time Distortion Based Scheduling for Delivery of Sealable Media , 2002 .

[3]  Bernd Girod,et al.  Rate-distortion optimized video streaming with rich acknowledgments , 2004, IS&T/SPIE Electronic Imaging.

[4]  Antonio Ortega,et al.  Joint Selection of Source and Channel Rate for VBR Video Transmission Under ATM Policing Constraints , 1997, IEEE J. Sel. Areas Commun..

[5]  Rui Zhang,et al.  End-to-end distortion estimation for RD-based robust delivery of pre-compressed video , 2001, Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256).

[6]  Laurent Massoulié,et al.  Network characteristics: modelling, measurements, and admission control , 2003, IWQoS'03.

[7]  Kannan Ramchandran,et al.  Compression and reliable transmission of digital image and video signals , 1999 .

[8]  Deborah Estrin,et al.  RAP: An end-to-end rate-based congestion control mechanism for realtime streams in the Internet , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[9]  Hayder Radha,et al.  Scalable Internet video using MPEG-4 , 1999, Signal Process. Image Commun..

[10]  Bernd Girod,et al.  Receiver-based rate-distortion optimized interactive streaming for scalable bitstreams of light fields , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[11]  Zixiang Xiong,et al.  Packet loss protection of embedded data with fast local search , 2002, Proceedings. International Conference on Image Processing.

[12]  Bernd Girod,et al.  Rate-distortion optimized streaming of compressed light fields , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[13]  B. Girod,et al.  Computing rate-distortion optimized policies for hybrid receiver/sender driven streaming of multimedia , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[14]  Sanjeev Mehrotra.  Multiple description coding using overcomplete linear expansions , 2000 .

[15]  Matthew Mathis,et al.  The macroscopic behavior of the TCP congestion avoidance algorithm , 1997, CCRV.

[16]  Qian Zhang,et al.  AN EFFICIENT TRANSPORT SCHEME FOR MULTIMEDIA OVER WIRELESS INTERNET , 2000 .

[17]  Qian Zhang,et al.  Resource allocation for audio and video streaming over the Internet , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[18]  Philip A. Chou,et al.  Cost-distortion optimized streaming media over DiffServ networks , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[19]  Hamid Jafarkhani,et al.  Joint source-channel coding of images for channels with feedback , 1998 .

[20]  John D. Villasenor,et al.  Priority dropping in network transmission of scalable video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[21]  Ya-Qin Zhang,et al.  Transporting real-time video over the Internet: challenges and approaches , 2000, Proceedings of the IEEE.

[22]  Kannan Ramchandran,et al.  Wireless image transmission using multiple-description based concatenated codes , 2000, Proceedings DCC 2000. Data Compression Conference.

[23]  Richard E. Ladner,et al.  Graceful degradation over packet erasure channels through forward error correction , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[24]  Albert Wang,et al.  Error control for receiver-driven layered multicast of audio and video , 2001, IEEE Trans. Multim..

[25]  Georg Carle,et al.  Survey of error recovery techniques for IP-based audio-visual multicast applications , 1997, IEEE Netw..

[26]  David W. Lin,et al.  Optimal Bit Allocation for Coding of Video Signals over ATM Networks , 1997, IEEE J. Sel. Areas Commun..

[27]  Amy R. Reibman,et al.  Transmission of streaming video over an EGPRS wireless network , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[28]  J. Wolfowitz,et al.  An Introduction to the Theory of Statistics , 1951, Nature.

[29]  Keith W. Ross,et al.  Optimal streaming of layer-encoded multimedia presentations , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[30]  Juan Carlos De Martin,et al.  Distortion-based packet marking for mpeg video transmission over diffserv networks , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[31]  Christos Papadopoulos,et al.  Retransmission-Based Error Control for Continuous Media Applications , 1996 .

[32]  Bernd Girod,et al.  Rate-distortion optimized packet scheduling and routing for media streaming with path diversity , 2003, Data Compression Conference, 2003. Proceedings. DCC 2003.

[33]  Thomas Stockhammer,et al.  Streaming video over variable bit-rate wireless channels , 2004, IEEE Transactions on Multimedia.

[34]  Gary J. Sullivan,et al.  Rate-distortion optimization for video compression , 1998, IEEE Signal Process. Mag..

[35]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[36]  Mark Handley,et al.  Equation-based congestion control for unicast applications , 2000, SIGCOMM.

[37]  John M. Danskin,et al.  Joint source and channel coding for image transmission over lossy packet networks , 1996, Optics & Photonics.

[38]  Amarnath Mukherjee,et al.  On the Dynamics and Significance of Low Frequency Components of Internet Load , 1992 .

[39]  Aggelos K. Katsaggelos,et al.  Joint source coding and packet classification for real-time video transmission over differentiated services networks , 2005, IEEE Transactions on Multimedia.

[40]  Nariman Farvardin,et al.  Progressive transmission of images over memoryless noisy channels , 2000, IEEE Journal on Selected Areas in Communications.

[41]  Laura Farinetti,et al.  Perceptual classification of MPEG video for Differentiated-Services communications , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[42]  Thomas Stockhammer,et al.  Progressive Texture Video Streaming for Lossy Packet Networks , 2001 .

[43]  Thierry Turletti,et al.  Experiments with a Layered Transmission Scheme over the Internet , 1997 .

[44]  Juan Carlos De Martin.  Source-driven packet marking for speech transmission over differentiated-services networks , 2001, ICASSP.

[45]  Philip A. Chou,et al.  Cost-distortion optimized caching of streaming media , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[46]  Kannan Ramchandran,et al.  Clustering source/channel rate allocations for receiver-driven multicast under a limited number of streams , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[47]  Antonio Ortega,et al.  Optimal trellis-based buffered compression and fast approximations , 1994, IEEE Trans. Image Process..

[48]  Rui Zhang,et al.  Optimized video streaming over lossy networks with real-time estimation of end-to-end distortion , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[49]  Bernd Girod,et al.  Rate-distortion optimized video streaming with multiple deadlines , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[50]  Antonio Ortega,et al.  Robust video communication by combining scalability and multiple description coding techniques , 2003, IS&T/SPIE Electronic Imaging.

[51]  Antonio Ortega,et al.  Algorithms for streaming, caching and storage of digital media , 2002 .

[52]  Eve A. Riskin,et al.  Optimal bit allocation via the generalized BFOS algorithm , 1991, IEEE Trans. Inf. Theory.

[53]  Jinho Choi,et al.  A stable feedback control of the buffer state using the controlled Lagrange multiplier method , 1994, IEEE Trans. Image Process..

[54]  Venkata N. Padmanabhan,et al.  Using Differentiated Services Mechanisms to Improve Network Protocol and Application Performance , 1999 .

[55]  Philip A. Chou,et al.  Optimal pruning with applications to tree-structured source coding and modeling , 1989, IEEE Trans. Inf. Theory.

[56]  Aggelos K. Katsaggelos,et al.  A novel cost-distortion optimization framework for video streaming over differentiated services networks , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[57]  Bernd Girod,et al.  Layered coding vs. multiple descriptions for video streaming over multiple paths , 2003, MULTIMEDIA '03.

[58]  Aria Nosratinia,et al.  Progressive source-channel coding of images over bursty error channels , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[59]  Jean C. Bolot,et al.  The Case for FEC-based Error Control for Packet Audio in the Internet , 1997 .

[60]  Bernd Girod,et al.  Rate-distortion optimized video streaming with adaptive playout , 2002, Proceedings. International Conference on Image Processing.

[61]  Agnieszka C. Miguel,et al.  SPIHT for generalized multiple description coding , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[62]  Richard E. Ladner,et al.  Approximately optimal assignment for unequal loss protection , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[63]  Xue Li,et al.  Layered video multicast with retransmission (LVMR): evaluation of error recovery schemes , 1997, Proceedings of 7th International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV '97).

[64]  Antonio Ortega,et al.  Multiple description coding via polyphase transform and selective quantization , 1998, Electronic Imaging.

[65]  Yao Wang,et al.  Transmission of multiple description and layered video over an EGPRS wireless network , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[66]  Bernd Girod,et al.  Rate-distortion optimized streaming from the edge of the network , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[67]  Yiwei Thomas Hou,et al.  On end-to-end architecture for transporting MPEG-4 video over the Internet , 2000, IEEE Trans. Circuits Syst. Video Technol..

[68]  Steven McCanne,et al.  Soft ARQ for Layered Streaming Media , 2001, J. VLSI Signal Process..

[69]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[70]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[71]  Antonio Ortega,et al.  Optimal segmentation of a VBR source for its parallel transmission over multiple ATM connections , 1997, Proceedings of International Conference on Image Processing.

[72]  Philip A. Chou,et al.  Application layer error-correction coding for rate-distortion optimized streaming to wireless clients , 2002, IEEE Transactions on Communications.

[73]  Albert Wang,et al.  FEC and pseudo-ARQ for receiver-driven layered multicast of audio and video , 2000, Proceedings DCC 2000. Data Compression Conference.

[74]  Madhu Sudan,et al.  Priority encoding transmission , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[75]  Avideh Zakhor,et al.  Internet video using error resilient scalable compression and cooperative transport protocol , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[76]  Sanjoy Paul.  Layered Video Multicast with Retransmission (LVMR) , 1998 .

[77]  Philip A. Chou,et al.  RaDiO Edge: Rate-Distortion Optimized Proxy-Driven Streaming From the Network Edge , 2006, IEEE/ACM Transactions on Networking.

[78]  Jean Bolot,et al.  Adaptive FEC-Based Error Control for Interactive Audio in the Internet , 1998 .

[79]  Juan Carlos De Martin,et al.  Delivery of MPEG video streams with constant perceptual quality of service , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[80]  Zhi-Li Zhang,et al.  Channel condition ARQ rate control for real-time wireless video under buffer constraints , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[81]  Douglas Comer,et al.  Internetworking with TCP/IP , 1988 .

[82]  Guijin Wang,et al.  Robust Scalable Video Streaming over Internet with Network-Adaptive Congestion Control and Unequal Loss Protection , 2001 .

[83]  Henning Schulzrinne,et al.  Dynamic QoS control of multimedia applications based on RTP , 1996, Comput. Commun..

[84]  Bernd Girod,et al.  R-D optimized media streaming enhanced with adaptive media playout , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[85]  Mark Handley,et al.  Equation-based congestion control for unicast applications , 2000, SIGCOMM 2000.

[86]  JongWon Kim,et al.  Relative priority based QoS interaction between video applications and differentiated service networks , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[87]  K. Ramchandran,et al.  Multiple description source coding using forward error correction codes , 1999, Conference Record of the Thirty-Third Asilomar Conference on Signals, Systems, and Computers (Cat. No.CH37020).

[88]  Jin Li,et al.  Scalable audio streaming over the internet with network-aware rate-distortion optimization , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[89]  Yair Shoham,et al.  Efficient bit allocation for an arbitrary set of quantizers [speech coding] , 1988, IEEE Trans. Acoust. Speech Signal Process..

[90]  Rui Zhang,et al.  Advances in channel-adaptive video streaming , 2002, Wirel. Commun. Mob. Comput..

[91]  Xiaolin Wu,et al.  Globally optimal uneven error-protected packetization of scalable code streams , 2002, Proceedings DCC 2002. Data Compression Conference.

[92]  JongWon Kim,et al.  Real-time encoding frame rate control for H.263+ video over the Internet , 1999, Signal Process. Image Commun..

[93]  Franklin A. Graybill,et al.  Introduction to the Theory of Statistics, 3rd ed. , 1974 .

[94]  Richard E. Ladner,et al.  Unequal loss protection: graceful degradation of image quality over packet erasure channels through forward error correction , 2000, IEEE Journal on Selected Areas in Communications.

[95]  A. Ortega,et al.  Optimal scheduling for streaming of scalable media , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[96]  Aggelos K. Katsaggelos,et al.  A rate-distortion optimized error control scheme for scalable video streaming over the Internet , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[97]  Gene Cheung,et al.  Directed acyclic graph based source modeling for data unit selection of streaming media over QoS networks , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[98]  Ali C. Begen,et al.  Rate-distortion optimized on-demand media streaming with server diversity , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[99]  Philip A. Chou,et al.  Computing rate-distortion optimized policies for streaming media to wireless clients , 2002, Proceedings DCC 2002. Data Compression Conference.

[100]  H. J. Larson,et al.  Introduction to the Theory of Statistics , 1973 .

[101]  Roger Fletcher,et al.  Practical methods of optimization; (2nd ed.) , 1987 .

[102]  Jean Cardinal,et al.  On the complexity of rate-distortion optimal streaming of packetized media , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.

[103]  Qian Zhang,et al.  Network-adaptive rate control with unequal loss protection for scalable video over Internet , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[104]  Philip A. Chou,et al.  RaDiO edge: rate-distortion optimized proxy-driven streaming from the network edge , 2006, TNET.

[105]  Henning Schulzrinne,et al.  Real Time Streaming Protocol (RTSP) , 1998, RFC.

[106]  Jill M. Boyce.  Packet loss resilient transmission of MPEG video over the Internet , 1999, Signal Process. Image Commun..

[107]  Donald F. Towsley,et al.  Adaptive FEC-based error control for Internet telephony , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[108]  James W. Modestino,et al.  Operational rate-distortion performance for joint source and channel coding of images , 1999, IEEE Trans. Image Process..

[109]  Steven McCanne,et al.  Simulation of FEC-based error control for packet audio on the Internet , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[110]  Bernd Girod,et al.  Robust Internet video transmission based on scalable coding and unequal error protection , 1999, Signal Process. Image Commun..