Incorporating Memory Management into User-Level Network Interfaces

User-level network interfaces allow applications direct access to the network without operating system intervention on every send and receive. Messages are transferred directly to and from user-space by the network interface while observing the traditional protection boundaries between processes. Current user-level network interfaces limit this message transfer to a per-process region of permanently-pinned physical memory to allow safe DMA. This approach is inflexible in that it requires data to be copied into and out of this memory region, and does not scale to a large number of processes. This paper presents an extension to the U-Net user-level network architecture (U-Net/MM) allowing messages to be transferred directly to and from any part of an application''s address space. This is achieved by integrating a translation look-aside buffer into the network interface and coordinating its operation with the operating system''s virtual memory subsystem. This mechanism allows network buffer pages to be pinned and unpinned dynamically. Two implementations of U-Net/MM are described, demonstrating that existing commodity hardware and commercial operating systems can efficiently support the architecture.

[1]  Larry L. Peterson,et al.  Fbufs: a high-bandwidth cross-domain transfer facility , 1994, SOSP '93.

[2]  Katherine Yelick,et al.  Introduction to Split-C , 1995 .

[3]  James R. Larus,et al.  Tempest and typhoon: user-level shared memory , 1994, ISCA '94.

[4]  William J. Dally,et al.  The M-machine multicomputer , 1997, Proceedings of the 28th Annual International Symposium on Microarchitecture.

[5]  Milon Mackey,et al.  An implementation of the Hamlyn sender-managed interface architecture , 1996, OSDI '96.

[6]  Jon Beecroft,et al.  Meiko CS-2 Interconnect Elan-Elite Design , 1994, Parallel Comput..

[7]  David E. Culler,et al.  The Explicit Token Store , 1990, J. Parallel Distributed Comput..

[8]  William J. Dally,et al.  The message-driven processor: a multicomputer processing node with efficient mechanisms , 1992, IEEE Micro.

[9]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[10]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[11]  Larry Rudolph,et al.  Message passing support on StarT-Voyager , 1998, Proceedings. Fifth International Conference on High Performance Computing (Cat. No. 98EX238).

[12]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[13]  Thorsten von Eicken,et al.  Low-Latency Communication over Fast Ethernet , 1996, Euro-Par, Vol. I.

[14]  Anoop Gupta,et al.  The Stanford FLASH multiprocessor , 1994, ISCA '94.

[15]  A. Chien,et al.  High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[16]  Kai Li,et al.  Virtual-Memory-Mapped Network Interfaces , 1995, IEEE Micro.