Optimizing memory-based messaging for scalable shared memory multiprocessor architectures

Passing messages between programs using shared memory, referred to as memory-based messaging, is a recognized technique for efficient communication that takes advantage of memory system performance. However, the conventional operating system support for this approach is inefficient, especially for large-scale shared memory multiprocessors, and is too complex to effectively support in hardware. This dissertation investigates the merits of memory-based messaging, with both hardware and software optimizations, that provides communication performance closely coupled with the memory system performance on a scalable memory architecture. The model, its implementation in an operating system kernel and hardware support for this approach in a scalable multiprocessor architecture is also described. The optimizations include address-valued signals, message-oriented memory consistency and automatic signaling on write. Address-valued signals integrate notification with the memory system, such that the sender can trigger a signal on an arbitrary memory address within a region of shared memory. Upon message arrival, the receiver is preempted by execution of its designated signal function. A single parameter is passed to the receiver's signal function, namely the receiver's shared memory address corresponding to the sender's signaling address. Message-oriented memory consistency defines a new memory consistency protocol that matches the semantics of messages. This allows the memory system to reduce the number of memory interconnection transactions. It also prevents sender and receiver stalls during transmission of the message data. Automatic signal- on-write allows the sending application to trigger a message send operation as a side effect of writing the message. This allows the sending of messages without trapping to the kernel. Results of this thesis show these extensions provide a three to five fold improvement in communication performance over a comparable software-only messaging implementation. Address-valued signaling and automatic signal-on-write reduce the number of instructions executed during the time-critical paths for a factor of three performance improvement. Message-oriented memory consistency provide further performance improvement by reducing the latency of messages sent through large-scale memory systems. Furthermore, the extensions decrease the complexity of the memory system hardware and the operating system kernel.