A Communication-Efficient Causal Broadcast Protocol

A causal broadcast ensures that messages are delivered to all nodes (processes) preserving causal relation of the messages. In this paper, we propose a causal broadcast protocol for distributed systems whose nodes are logically organized in a virtual hypercube-like topology called VCube. Messages are broadcast by dynamically building spanning trees rooted in the message's source node. By using multiple trees, the contention bottleneck problem of a single root spanning tree approach is avoided. Furthermore, different trees can intersect at some node. Hence, by taking advantage of both the out-of-order reception of causally related messages at a node and these paths intersections, a node can delay to one or more of its children in the tree, the forwarding of the messages whose some causal dependencies it knows that the children in question can not satisfy yet. Such a delay does not induce any overhead. Experimental evaluation conducted on top of PeerSim simulator confirms the communication effectiveness of our causal broadcast protocol in terms of latency and message traffic reduction.

[1]  Mukesh Singhal,et al.  Broadcast with time and causality constraints for multimedia applications , 1996, Proceedings of EUROMICRO 96. 22nd Euromicro Conference. Beyond 2000: Hardware and Software Design Strategies.

[2]  Jon Postel,et al.  Internet Protocol , 1981, RFC.

[3]  Colin J. Fidge,et al.  Timestamps in Message-Passing Systems That Preserve the Partial Ordering , 1988 .

[4]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[5]  Mukesh Singhal,et al.  Real‐time causal message ordering in multimedia systems , 1997, Telecommun. Syst..

[6]  Nael B. Abu-Ghazaleh,et al.  Optimizing communication in time-warp simulators , 1998, Workshop on Parallel and Distributed Simulation.

[7]  Ajay D. Kshemkalyani,et al.  An Efficient Implementation of Vector Clocks , 1992, Inf. Process. Lett..

[8]  Mukesh Singhal,et al.  An efficient causal ordering algorithm for mobile computing environments , 1996, Proceedings of 16th International Conference on Distributed Computing Systems.

[9]  Xiaohua Jia,et al.  An algorithm to construct independent spanning trees on parity cubes , 2012, Theor. Comput. Sci..

[10]  Achour Mostéfaoui,et al.  Causal consistency: beyond memory , 2016, PPoPP.

[11]  Bobby Bhattacharjee,et al.  Triangle inequality variations in the internet , 2009, IMC '09.

[12]  B. R. Badrinath,et al.  Recording Distributed Snapshots Based on Causal Order of Message Delivery , 1992, Inf. Process. Lett..

[13]  André Schiper,et al.  A New Algorithm to Implement Causal Ordering , 1989, WDAG.

[14]  Noha Adly,et al.  Maintaining Causal Order in Large Scale Distributed Systems Using a Logical Hierarchy , 1995 .

[15]  Bo Zhang,et al.  Towards network triangle inequality violation aware distributed systems , 2007, IMC '07.

[16]  Stéphane Weiss,et al.  A Probabilistic Causal Message Ordering Mechanism , 2017 .

[17]  David Simplot-Ryl,et al.  DHT-based distributed ALE engine in RFID middleware , 2011, 2011 IEEE International Conference on RFID-Technologies and Applications.

[18]  Hrushikesha Mohanty,et al.  Causal ordering in event notification service systems for mobile users , 2004, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004..

[19]  Jim Kurose,et al.  Computer Networking: A Top-Down Approach (6th Edition) , 2007 .

[20]  Sophia Drossopoulou,et al.  Tree topologies for causal message delivery , 2017, AGERE!@SPLASH.

[21]  Elias Procópio Duarte,et al.  VCube: A Provably Scalable Distributed Diagnosis Algorithm , 2014, 2014 5th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems.

[22]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[23]  Mustaque Ahamad,et al.  Plausible clocks: constant size logical clocks for distributed systems , 1996, Distributed Computing.

[24]  Nalini Venkatasubramanian,et al.  FaReCast: Fast, Reliable Application Layer Multicast for Flash Dissemination , 2010, Middleware.

[25]  Anjali Gupta,et al.  Efficient Routing for Peer-to-Peer Overlays , 2004, NSDI.

[26]  Murat Demirbas,et al.  The impact of data aggregation on the performance of wireless sensor networks , 2008, Wirel. Commun. Mob. Comput..

[27]  Richard D. Schlichting,et al.  Fault-Tolerant Broadcasts , 1984, Sci. Comput. Program..

[28]  Bobby Bhattacharjee,et al.  Decentralized Message Ordering for Publish/Subscribe Systems , 2006, Middleware.

[29]  Tilman Wolf,et al.  Characterizing network processing delay , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[30]  Pierre Sens,et al.  A Publish/Subscribe System Using Causal Broadcast over Dynamically Built Spanning Trees , 2017, 2017 29th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD).

[31]  Friedemann Mattern,et al.  Virtual Time and Global States of Distributed Systems , 2002 .

[32]  Ali Ghodsi,et al.  The potential dangers of causal consistency and an explicit solution , 2012, SoCC '12.

[33]  Takumi Ohba,et al.  IoT Network Architecture Using Packet Aggregation and Disaggregation , 2016, 2016 5th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI).

[34]  Bernadette Charron-Bost,et al.  Concerning the Size of Logical Clocks in Distributed Systems , 1991, Inf. Process. Lett..

[35]  Achour Mostéfaoui,et al.  Probabilistic Causal Message Ordering , 2017, PaCT.

[36]  Kazuyuki Shudo,et al.  Message bundling on structured overlays , 2017, 2017 IEEE Symposium on Computers and Communications (ISCC).

[37]  Pierre Sens,et al.  An Aggregation-Based Routing Protocol for Structured Peer to Peer Overlay Networks , 2010, AP2PS 2010.

[38]  Herman Arnold Engelbrecht,et al.  A Survey of State Persistency in Peer-to-Peer Massively Multiplayer Online Games , 2012, IEEE Transactions on Parallel and Distributed Systems.

[39]  Márk Jelasity,et al.  PeerSim: A scalable P2P simulator , 2009, 2009 IEEE Ninth International Conference on Peer-to-Peer Computing.

[40]  Mustaque Ahamad,et al.  Implementing and programming causal distributed shared memory , 1991, [1991] Proceedings. 11th International Conference on Distributed Computing Systems.

[41]  Jim Kurose,et al.  Computer Networking: A Top-Down Approach , 1999 .

[42]  André Schiper,et al.  Lightweight causal and atomic group multicast , 1991, TOCS.

[43]  Kenneth P. Birman,et al.  Reliable communication in the presence of failures , 1987, TOCS.

[44]  Gérard Padiou,et al.  A coordination-level middleware for supporting flexible consistency in CSCW , 2006, 14th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP'06).