KStreams: kernel support for efficient data streaming in proxy servers

Growth in broadband connectivity is making media streaming applications increasingly popular. For scalability, media is streamed across sets of proxy servers embedded in overlay networks, where the quality of delivered content depends both on available network capacities across overlay nodes and the capabilities of proxy servers. This paper addresses proxy server performance for media streaming and for the delivery of live media content. Our approach to efficient content delivery is to develop a set of kernel-level data streaming abstractions, termed KStreams. Compared to user-level solutions, KStreams (1) offers improved performance for the multiple data forwarding models commonly used in data distribution networks, (2) reduces per stream overheads by eliminating unnecessary system calls and memory copying, and (3) offers improved levels of predictability for the Quality of Service (QoS) experienced by media streams due to its use of non-preemptable kernel-level threads and its ability to directly interact with the CPU scheduler and other kernel-level resource managers. (4) Once initiated, KStreams operates without further involvement of and asynchronously to applications, permitting them to carry out other tasks.

[1]  Karsten Schwan,et al.  Lightweight Morphing Support for Evolving Middleware Data Exchanges in Distributed Applications , 2005, 25th IEEE International Conference on Distributed Computing Systems (ICDCS'05).

[2]  Christian Poellabauer,et al.  Resource-aware stream management with the customizable dproc distributed monitoring mechanisms , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[3]  Karsten Schwan,et al.  The active streams approach to adaptive distributed applications and services , 2001 .

[4]  Karsten Schwan,et al.  IQ-RUDP: coordinating application adaptation with network transport , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[5]  Verena Kahmann,et al.  A proxy architecture for collaborative media streaming , 2002 .

[6]  Prashant J. Shenoy,et al.  AMPS: a flexible, scalable proxy testbed for implementing streaming services , 2004, NOSSDAV '04.

[7]  Karsten Schwan,et al.  KStreams: Kernel Support for Efficient End-to-End Data Streaming , 2004 .

[8]  Aravind Srinivasan,et al.  Scalable resilient media streaming , 2004, NOSSDAV '04.

[9]  Christian Poellabauer,et al.  Analysis of a window-constrained scheduler for real-time and best-effort packet streams , 2000, Proceedings 21st IEEE Real-Time Systems Symposium.

[10]  Christos Papadopoulos,et al.  An adaptive multiple retransmission technique for continuous media streams , 2004, NOSSDAV '04.

[11]  Karsten Schwan,et al.  Quality events: a flexible mechanism for quality of service management , 2001, Proceedings Seventh IEEE Real-Time Technology and Applications Symposium.

[12]  Steven McCanne,et al.  An active service framework and its application to real-time multimedia transcoding , 1998, SIGCOMM '98.

[13]  S. Lam,et al.  An Architecture towards Eecient Os Support for Distributed Multimedia , 1996 .

[14]  Judy Kay,et al.  A fair share scheduler , 1988, CACM.

[15]  Christian Poellabauer,et al.  Lightweight kernel/user communication for real-time and multimedia applications , 2001, NOSSDAV '01.

[16]  Daniela Rosu,et al.  An evaluation of TCP splice benefits in web proxy servers , 2002, WWW.

[17]  Robert Tappan Morris,et al.  Resilient overlay networks , 2001, SOSP.

[18]  Reza Rejaie,et al.  Mocha: a quality adaptive multimedia proxy cache for internet streaming , 2001, NOSSDAV '01.

[19]  William Thies,et al.  StreamIt: A Language for Streaming Applications , 2002, CC.

[20]  Guru M. Parulkar,et al.  Design of Universal Continuous Media I/O , 1995, NOSSDAV.

[21]  Donald F. Towsley,et al.  Optimal proxy cache allocation for efficient streaming media distribution , 2002, IEEE Transactions on Multimedia.

[22]  A Survey of the History of Internet Multicast , 2000 .

[23]  Kirk L. Johnson,et al.  Overcast: reliable multicasting with on overlay network , 2000, OSDI.

[24]  Karsten Schwan,et al.  Kernel Plugins: When a VM Is Too Much , 2004, Virtual Machine Research and Technology Symposium.

[25]  Yousef A. Khalidi,et al.  An Efficient Zero-Copy I/O Framework for UNIX , 1995 .

[26]  José Carlos Brustoloni,et al.  Interoperation of copy avoidance in network and file I/O , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[27]  Zhi-Li Zhang,et al.  A network-conscious approach to end-to-end video delivery over wide area networks using proxy servers , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[28]  David A. Maltz,et al.  TCP Splice for application layer proxy performance , 1999, J. High Speed Networks.

[29]  Willy Zwaenepoel,et al.  IO-Lite: a unified I/O buffering and caching system , 1999, TOCS.

[30]  Lars C. Wolf,et al.  A proxy architecture for collaborative media streaming , 2001, M3W.

[31]  Joseph Pasquale,et al.  Container shipping: operating system support for I/O-intensive applications , 1994, Computer.

[32]  Greg Eisenhauer.  Portable Self-Describing Binary Data Streams , 1994 .

[33]  Wu-chi Feng,et al.  Quality-adaptive media streaming by priority drop , 2003, NOSSDAV '03.

[34]  Eric A. Brewer,et al.  Cluster-based scalable network services , 1997, SOSP.

[35]  Larry L. Peterson,et al.  Fbufs: a high-bandwidth cross-domain transfer facility , 1994, SOSP '93.

[36]  Donald F. Towsley,et al.  Proxy prefix caching for multimedia streams , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[37]  Christian Poellabauer,et al.  Cooperative run-time management of adaptive applications and distributed resources , 2002, MULTIMEDIA '02.

[38]  Joseph Pasquale,et al.  Exploiting In-Kernel Data Paths to Improve I/O Throughput and CPU Availability , 1993, USENIX Winter.

[39]  Bobby Bhattacharjee,et al.  Scalable application layer multicast , 2002, SIGCOMM '02.

[40]  Karsten Schwan,et al.  SmartPointers: Personalized Scientific Data Portals In Your Hand , 2002, ACM/IEEE SC 2002 Conference (SC'02).