HiPEC: high performance external virtual memory caching

Traditional operating systems use a fixed LRU-like page replacement policy and centralized frame pool that cannot properly serve all types of memory access patterns of various applications As a result many memory-intensive applications such as databases multimedia applications and scientific simulators induce excessive page faults and page replacement when running on top of existing operating systems. This paper presents a High Performance External virtual memory Caching mechanism (HiPEC) to provide applications with their own specific page replacement management. The user specific policy programmed in the HiPEC command set is stored in user address space. When a page fault occurs the kernel fetches and interprets the corresponding policy commands to perform the user specific page replacement management. Experimental results show that HiPEC induces little overhead and can significantly improve performance for memory-intensive applications.

[1]  Brian N. Bershad,et al.  Scheduler activations: effective kernel support for the user-level management of parallelism , 1991, TOCS.

[2]  John Wilkes,et al.  An introduction to disk drive modeling , 1994, Computer.

[3]  Goetz Graefe,et al.  Query evaluation techniques for large databases , 1993, CSUR.

[4]  Amin Vahdat,et al.  Tools for the development of application-specific virtual memory management , 1993, OOPSLA '93.

[5]  Brian N. Bershad,et al.  User-level interprocess communication for shared memory multiprocessors , 1991, TOCS.

[6]  Michael L. Scott,et al.  Simple but effective techniques for NUMA memory management , 1989, SOSP '89.

[7]  Jeffrey D. Mcdonald.  Particle simulation in a multiprocessor environment , 1991 .

[8]  GraefeGoetz.  Query evaluation techniques for large databases , 1993 .

[9]  David L. Black,et al.  The duality of memory and communication in the implementation of a multiprocessor operating system , 1987, SOSP '87.

[10]  David R. Cheriton,et al.  The V distributed system , 1988, CACM.

[11]  Michael Stonebraker,et al.  Operating system support for database management , 1981, CACM.

[12]  Jeffrey C. Mogul,et al.  The packer filter: an efficient mechanism for user-level network code , 1987, SOSP '87.

[13]  Richard F. Rashid,et al.  Zone Garbage Collection , 1990, USENIX MACH Symposium.

[14]  David L. Black,et al.  Scheduling and resource management techniques for multiprocessors , 1990 .

[15]  Steven McCanne,et al.  The BSD Packet Filter: A New Architecture for User-level Packet Capture , 1993, USENIX Winter.

[16]  Brian N. Bershad,et al.  Efficient Packet Demultiplexing for Multiple Endpoints and Large Messages , 1994, USENIX Winter.

[17]  Richard Draves,et al.  Page Replacement and Reference Bit Emulation in Mach , 1991, USENIX MACH Symposium.

[18]  Brian N. Bershad,et al.  Using continuations to implement thread management and communication in operating systems , 1991, SOSP '91.

[19]  William J. Bolosky,et al.  Mach: A New Kernel Foundation for UNIX Development , 1986, USENIX Summer.

[20]  Michael N. Nelson,et al.  A Flexible External Paging Interface , 1993, USENIX Microkernels and Other Kernel Architectures Symposium.

[21]  Richard Draves,et al.  Moving the Default Memory Manager Out of the Mach Kernel , 1991, USENIX MACH Symposium.

[22]  Yoonho Park,et al.  User-Level Physical Memory Management for Mach , 1991, USENIX MACH Symposium.

[23]  Dylan McNamee,et al.  Extending the Mach External Pager Interface to Accomodate User-Level Page Replacement Policies , 1990, USENIX MACH Symposium.

[24]  Michael Wayne Young.  Exporting a user interface to memory management from a communication-oriented operating system , 1989 .

[25]  Hendrik A. Goosen,et al.  Paradigm: a highly scalable shared-memory multicomputer architecture , 1991, Computer.

[26]  Gerald W. Neufeld,et al.  User Level IPC and Device Management in the Raven Kernel , 1993, USENIX Microkernels and Other Kernel Architectures Symposium.

[27]  David R. Cheriton,et al.  Application-controlled physical memory using external page-cache management , 1992, ASPLOS V.

[28]  Joseph Pasquale,et al.  A high performance multi-structured file system design , 1991, SOSP '91.

[29]  Emin Gün Sirer,et al.  SPIN: an extensible microkernel for application-specific operating system services , 1994, EW 6.

[30]  Ruei-Chuan Chang,et al.  An implementation of an external pager interface on BSD UNIX , 1995, J. Syst. Softw..

[31]  David R. Cheriton,et al.  A market approach to operating system memory allocation , 1996 .

[32]  Anoop Gupta,et al.  The DASH prototype: implementation and performance , 1992, ISCA '92.