Enabling lock-free concurrent workers over temporal graphs composed of multiple time-series

Time series are commonly used to store temporal data, e.g., sensor measurements. However, when it comes to complex analytics and learning tasks, these measurements have to be combined with structural context data. Temporal graphs, connecting multiple time-series, have proven to be very suitable to organize such data and ultimately empower analytic algorithms. Computationally intensive tasks often need to be distributed and parallelized among different workers. For tasks that cannot be split into independent parts, several workers have to concurrently read and update these shared temporal graphs. This leads to inconsistency risks, especially in the case of frequent updates. Distributed locks can mitigate these risks but come with a very high-performance cost. In this paper, we present a lock-free approach allowing to concurrently modify temporal graphs. Our approach is based on a composition operator able to do online reconciliation of concurrent modifications of temporal graphs. We evaluate the efficiency and scalability of our approach compared to lock-based approaches.

[1]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[2]  John K. Ousterhout,et al.  In Search of an Understandable Consensus Algorithm , 2014, USENIX ATC.

[3]  Tancred Lindholm,et al.  A three-way merge for XML documents , 2004, DocEng '04.

[4]  Barbara Liskov,et al.  Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions , 1999 .

[5]  Colin J. Fidge,et al.  Logical time in distributed computing systems , 1991, Computer.

[6]  Marc Shapiro,et al.  Conflict-Free Replicated Data Types , 2011, SSS.

[7]  Ion Stoica,et al.  Time-evolving graph processing at scale , 2016, GRADES '16.

[8]  Marie-Pierre Gervais,et al.  Modeling Services and Web Services: Application of ModelBus , 2005, Software Engineering Research and Practice.

[9]  Abraham Silberschatz,et al.  An optimistic commit protocol for distributed transaction management , 1991, SIGMOD '91.

[10]  Mukesh Singhal,et al.  Logical Time: Capturing Causality in Distributed Systems , 1996, Computer.

[11]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[12]  Wenguang Chen,et al.  Chronos: a graph engine for temporal graph analysis , 2014, EuroSys '14.

[13]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[14]  Ethan V. Munson,et al.  Using versioned tree data structure, change detection and node identity for three-way XML merging , 2010, DocEng '10.

[15]  Jacques Klein,et al.  Beyond discrete modeling: A continuous and efficient model for IoT , 2015, 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS).

[16]  Jacques Klein,et al.  Model-Driven Analytics: Connecting Data, Domain Knowledge, and Learning , 2017, ArXiv.

[17]  Olga Baysal,et al.  diffX: an algorithm to detect changes in multi-version XML documents , 2005, CASCON.

[18]  Yves Le Traon,et al.  Analyzing Complex Data in Motion at Scale with Temporal Graphs , 2017, SEKE.

[19]  Benjamin Reed,et al.  Omid: Lock-free transactional support for distributed data stores , 2014, 2014 IEEE 30th International Conference on Data Engineering.

[20]  Wilfred Ng,et al.  Repairing Inconsistent Merged XML Data , 2003, DEXA.

[21]  Joseph M. Hellerstein,et al.  GraphLab: A New Framework For Parallel Machine Learning , 2010, UAI.

[22]  Bruce G. Lindsay,et al.  Efficient commit protocols for the tree of processes model of distributed transactions , 1985, OPSR.

[23]  Naohiro Hayashibara,et al.  ChainVoxel: A Data Structure for Scalable Distributed Collaborative Editing for 3D Models , 2016, 2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech).

[24]  Talel Abdessalem,et al.  A probabilistic XML merging tool , 2011, EDBT/ICDT '11.

[25]  Marie-Pierre Gervais,et al.  Model Bus: Towards the Interoperability of Modelling Tools , 2003, MDAFA.

[26]  Marc Shapiro,et al.  Designing a commutative replicated data type , 2007, ArXiv.

[27]  Daniel J. Abadi,et al.  Calvin: fast distributed transactions for partitioned database systems , 2012, SIGMOD Conference.

[28]  Carlos Guestrin,et al.  Distributed GraphLab : A Framework for Machine Learning and Data Mining in the Cloud , 2012 .