Network-Centric Buffer Cache Organization

A pass-through server such as an NFS server backed by an iSCSI (Internet Small Computer System Interface) storage server only passes data between the storage server and NFS clients. Ideally it should require at most one data copying operation on sending or receiving, as in normal IP routers. In practice, pass-through servers actually incur multiple data copying operations because they are implemented using a layered architecture where each layer has its own internal data representation. This paper describes the design, implementation and evaluation of a novel network-centric buffer-caching scheme called NCache that minimizes data copying overhead in pass-through servers without requiring significant changes to their existing implementation. By organizing the data being passed or cached in a network friendly format, NCache is able to eliminate all unnecessary data copying. The key innovation in NCache is that it exploits the fact that pass-through servers do not interpret data, by replacing physical copying with logical copying in a way transparent to existing software. This transparency enables NCache to be easily portable to many operating systems. The authors have successfully built a Linux-based NCache prototype that can be applied to in kernel NFS and static Web servers. Empirical performance measurements collected from this prototype show that by reducing the CPU load associated with data copying, NCache is able to provide up to 92% improvement in throughput for NFS server and up to 47% for Web server

[1]  Margo I. Seltzer,et al.  Structure and Performance of the Direct Access File System , 2002, USENIX ATC, General Track.

[2]  Tzi-cker Chiueh,et al.  An NFS Trace Player for File System Evaluation , 2003 .

[3]  Peter F. Corbett,et al.  The Direct Access File System , 2003, FAST.

[4]  Hsiao-Keng Jerry Chu,et al.  Zero-Copy TCP in Solaris , 1996, USENIX Annual Technical Conference.

[5]  Jeffrey S. Chase,et al.  Payload Caching: High-Speed Data Forwarding for Network Intermediaries , 2001, USENIX ATC, General Track.

[6]  José Carlos Brustoloni,et al.  Effects of buffering semantics on I/O performance , 1996, OSDI '96.

[7]  Willy Zwaenepoel,et al.  IO-Lite: a unified I/O buffering and caching system , 1999, TOCS.

[8]  Julian Satran,et al.  Internet Small Computer Systems Interface (iSCSI) , 2004, RFC.

[9]  Larry L. Peterson,et al.  Fbufs: a high-bandwidth cross-domain transfer facility , 1994, SOSP '93.

[10]  Dawson R. Engler,et al.  Exokernel: an operating system architecture for application-level resource management , 1995, SOSP.

[11]  Jim Zelenka,et al.  A cost-effective, high-bandwidth storage architecture , 1998, ASPLOS VIII.

[12]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[13]  Larry L. Peterson,et al.  Making paths explicit in the Scout operating system , 1996, OSDI '96.

[14]  Gang Peng,et al.  A case for network-centric buffer cache organization , 2003, 11th Symposium on High Performance Interconnects, 2003. Proceedings..