A Flexible Causal Broadcast Communication Interface for Distributed Applications

Abstract The paper describes a causal broadcast communication interface that allows distributed applications to flexibly and uniformly specify message ordering requirements. The interface, implemented by the communication layer (or kernel), allows event-driven programming whereby application entities specify partial ordering on occurrence of messages, and all entities perceive the flow of logical time by exchanging messages in the specified order. Since entities see the same order on messages, they have the same view of the application state at every tick in logical time. The paper encourages an explicit representation of causal relationships among messages, whereby a message carries ordering constraints in the form of predicates on the delivery of other messages and a message is delivered at destinations after its predicates are satisfied. This provides flexibility in the communication structure of applications, whereby: (1) weak and strong orderings may be specified uniformly and (2) complex ordering relationship can be captured (e.g., nondeterminism in application). In addition, the concurrency inherent in the application can be precisely extracted and exploited by the kernel message delivery protocols. The broadcast interface integrates this style of causal ordering into a “process group” based program execution model that allows application entities to organize themselves as members of a group and exchange messages within the group in the presence of members joining and leaving the group. The paper describes replication mechanisms built using the interface for remote procedure call and distributed shared memory.