Active message applications programming interface and communication subsystem organization

High-performance network hardware is advancing, with multi-gigabit link bandwidths and sub-microsecond switch latencies. Network-interface hardware also continues to evolve, although the design space remains lar ge and diverse. One critical abstraction, a simple, portable, and general-purpose communications interface, is required to make ef fective use of these increasingly high-performance networks and their capable interfaces. Without a new interface, the software overheads of existing ones will dominate communication costs, and many applications may not benefit from the advancements in network hardware. This document specifies a new active message communications interface for these networks. Its primitives, in essence an instruction set for communications, map ef ficiently onto underlying network hardware and compose ef fectively into higher -level protocols and applications. For high-performance implementations, the interface enables direct application -network interface interactions. In the common case, for applications exhibiting locality in communication, these interactions bypass the operating system. T o nable the construction of lar ge-scale, general-purpose systems, the interface supports the protected multiprogramming of many users onto finite network resources.This document also describes a prototype system that uses the virtual-memory facilities of the Solaris operating system to implement virtual networks that support protected, network multiprogramming. The system caches the active communication endpoints in network-interface memory and demand-pages them between the host and network-interface memories.