Active Message Applications Programming Interface and Communication Subsystem Organization
Mainwaring, Alan M.
Culler, David E.
Technical Report Identifier: CSD-96-918
Abstract: High-performance network hardware is advancing, with multi-gigabit link bandwidths and sub-microsecond switch latencies. Network-interface hardware also continues to evolve, although the design space remains large and diverse. One critical abstraction, a simple, portable, and general-purpose communications interface, is required to make effective use of these increasingly high-performance networks and their capable interfaces. Without a new interface, the software overheads of existing ones will dominate communication costs, and many applications may not benefit from the advancements in network hardware.
This document specifies a new active message communications interface for these networks. Its primitives, in essence an instruction set for communications, map efficiently onto underlying network hardware and compose effectively into higher-level protocols and applications. For high-performance implementations, the interface enables direct application-network interface interactions. In the common case, for applications exhibiting locality in communication, these interactions bypass the operating system. To enable the construction of large-scale, general-purpose systems, the interface supports the protected multiprogramming of many users onto finite network resources. This document also describes a prototype system that uses the virtual-memory facilities of the Solaris operating system to implement virtual networks that support protected, network multiprogramming. The system caches the active communication endpoints in network-interface memory and demand-pages them between the host and network-interface memories.