Diamond: Automating Data Management and Storage for Wide-Area, Reactive Applications

Users of today's popular wide-area apps (e.g., Twitter, Google Docs, and Words with Friends) must no longer save and reload when updating shared data; instead, these applications are reactive, providing the illusion of continuous synchronization across mobile devices and the cloud. Achieving this illusion poses a complex distributed data management problem for programmers. This paper presents the first reactive data management service, called Diamond, which provides persistent cloud storage, reliable synchronization between storage and mobile devices, and automated execution of application code in response to shared data updates. We demonstrate that Diamond greatly simplifies the design of reactive applications, strengthens distributed data sharing guarantees, and supports automated reactivity with low performance overhead.

[1]  Willy Zwaenepoel,et al.  Munin: distributed shared memory based on type-specific memory coherence , 1990, PPOPP '90.

[2]  Robert Grimm,et al.  PADS: A Policy Architecture for Distributed Storage Systems , 2009, NSDI.

[3]  Wilson C. Hsieh,et al.  Bigtable: A Distributed Storage System for Structured Data , 2006, TOCS.

[4]  Lei Gao,et al.  Serving large-scale batch computed data with project Voldemort , 2012, FAST.

[5]  Santosh Krishnan,et al.  Google Compute Engine , 2015 .

[6]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[7]  Maurice Herlihy.  Optimistic concurrency control for abstract data types , 1986, PODC '86.

[8]  Roy Fielding,et al.  Architectural Styles and the Design of Network-based Software Architectures"; Doctoral dissertation , 2000 .

[9]  Harsha V. Madhyastha,et al.  Simba: tunable end-to-end data consistency for mobile apps , 2015, EuroSys.

[10]  Sharma Chakravarthy.  Sentinel: an object-oriented DBMS with event-based rules , 1997, SIGMOD '97.

[11]  Robert Tappan Morris,et al.  Eyo: Device-Transparent Personal Storage , 2011, USENIX Annual Technical Conference.

[12]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[13]  Paul Hudak,et al.  Functional reactive programming from first principles , 2000, PLDI '00.

[14]  Frank Wm. Tompa,et al.  Efficiently updating materialized views , 1986, SIGMOD '86.

[15]  Gregory Cooper,et al.  Thialfi: a client notification service for internet-scale applications , 2011, SOSP '11.

[16]  Robert Tappan Morris,et al.  Flexible, Wide-Area Storage for Distributed Systems with WheelFS , 2009, NSDI.

[17]  Lei Gao,et al.  PRACTI Replication , 2006, NSDI.

[18]  Arvind Krishnamurthy,et al.  Building consistent transactions with inconsistent replication , 2015, SOSP.

[19]  Barbara Liskov,et al.  Viewstamped Replication: A New Primary Copy Method to Support Highly-Available Distributed Systems , 1999, PODC '88.

[20]  Eddie Kohler,et al.  Type-aware transactions for faster concurrent code , 2016, EuroSys.

[21]  William E. Weihl,et al.  Local atomicity properties: modular concurrency control for abstract data types , 1989, TOPL.

[22]  David G. Andersen,et al.  There is more consensus in Egalitarian parliaments , 2013, SOSP.

[23]  Narain H. Gehani,et al.  The Ode active database: trigger semantics and implementation , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[24]  Marc Shapiro,et al.  Conflict-Free Replicated Data Types , 2011, SSS.

[25]  Krzysztof R. Apt,et al.  Principles of constraint programming , 2003 .

[26]  Yawei Li,et al.  Megastore: Providing Scalable, Highly Available Storage for Interactive Services , 2011, CIDR.

[27]  Prashant Malik,et al.  Cassandra: a decentralized structured storage system , 2010, OPSR.

[28]  Maurice Herlihy,et al.  Apologizing versus asking permission: optimistic concurrency control for abstract data types , 1990, TODS.

[29]  David Mazières,et al.  A low-bandwidth network file system , 2001, SOSP.

[30]  Samuel Madden,et al.  Transactional Consistency and Automatic Management in an Application Data Cache , 2010, OSDI.

[31]  Sandeep Koranne,et al.  Boost C++ Libraries , 2011 .

[32]  James Goodwill,et al.  The Swift Programming Language , 2015 .

[33]  Mahadev Satyanarayanan,et al.  Disconnected Operation in the Coda File System , 1999, Mobidata.

[34]  Paul Hudak,et al.  Memory coherence in shared virtual memory systems , 1989, TOCS.

[35]  Michael Stonebraker,et al.  Readings in Database Systems , 1988 .

[36]  Guido Salvaneschi,et al.  We have a DREAM: distributed reactive programming with consistency guarantees , 2014, DEBS '14.

[37]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[38]  Arvind Krishnamurthy,et al.  Customizable and Extensible Deployment for Mobile/Cloud Applications , 2014, OSDI.

[39]  Tim Kraska,et al.  MDCC: multi-data center consistency , 2012, EuroSys '13.