Anemone: adaptive network memory engine

There is a constant battle to break-even between continuing improvements in DRAM capacities and the demands for even more memory by modern memory-intensive high-performance applications. Such applications do not take long to hit the physical memory limit and start paging to disk, which in turn considerably slows down their performance. We tackle this problem in the Adaptive Network Memory Engine (Anemone) project by pooling together the distributed memory resources of multiple machines across a gigabit network based cluster. Anemone is a distributed memory virtualization system that can dramatically improve application performance by paging over the gigabit network to the unused memory of remote clients. Anemone provides clients with completely transparent access to a potentially unlimited amount of collective memory pool. Earlier research efforts in this area advocate significant modifications to client systems, either in terms of a specific programming interface for applications, or in terms of extensive changes to the operating systems and device drivers. In contrast, Anemone requires no modifications to either the client system or the memory-intensive application.

[1]  Li Xiao,et al.  Incorporating job migration and network RAM to share cluster memory resources , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[2]  Rob Williams,et al.  Linux device drivers , 2006 .

[3]  Francisco Brasileiro,et al.  Using Remote Memory to Stabilise Data EÆciently on an EXT2 Linux File System , 2002 .

[4]  David E. Culler,et al.  A case for NOW (networks of workstation) , 1995, PODC '95.

[5]  Robin Fairbairns,et al.  The Design and Implementation of an Operating System to Support Distributed Multimedia Applications , 1996, IEEE J. Sel. Areas Commun..

[6]  Thu D. Nguyen,et al.  Cooperative caching middleware for cluster-based servers , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[7]  Sandeep K. Gupta,et al.  AVM: application-level virtual memory , 1995, Proceedings 5th Workshop on Hot Topics in Operating Systems (HotOS-V).

[8]  Evangelos P. Markatos,et al.  The Network RamDisk: Using remote memory on heterogeneous NOWs , 1999, Cluster Computing.

[9]  Evangelos P. Markatos,et al.  Implementation of a Reliable Remote Memory Pager , 1996, USENIX ATC.

[10]  Jim Griffioen,et al.  A New Design for Distributed Systems: The Remote Memory Model , 1990, USENIX Summer.

[11]  Matteo Frigo,et al.  Cache-oblivious algorithms , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[12]  Greg Kroah-Hartman,et al.  Linux Device Drivers , 1998 .

[13]  Carl Smith,et al.  NFS Version 3: Design and Implementation , 1994, USENIX Summer.

[14]  Joel F. Bartlett,et al.  A NonStop kernel , 1981, SOSP.

[15]  Ian Alan McDonald Remote Paging in a Single Address Space Operating System supporting Quality of Service , 1999 .

[16]  Sanjeev Setia,et al.  Dodo: a user-level system for exploiting idle memory in workstation clusters , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[17]  Nikolaos Hardavellas,et al.  Cashmere-VLM: Remote memory paging for software distributed shared memory , 1999, Proceedings 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing. IPPS/SPDP 1999.

[18]  Daniel Pierre Bovet,et al.  Understanding the Linux Kernel , 2000 .

[19]  David Clark,et al.  Architectural considerations for a new generation of protocols , 1990, SIGCOMM 1990.

[20]  David E. Culler,et al.  A case for NOW (networks of workstation) , 1995, PODC '95.

[21]  Anna R. Karlin,et al.  Implementing global memory management in a workstation cluster , 1995, SOSP.