Syndicate: virtual cloud storage through provider composition

Syndicate is a storage service that builds a coherent storage abstraction from already-deployed commodity components, including cloud storage, edge caches, and dataset providers. It is unique in that it not only offers consistent semantics across multiple providers, but also offers a flexible programming model to applications so they can define their own provider-agnostic storage functionality. In doing so, Syndicate fully decouples applications from providers, allowing applications to choose them based on how well they enhance data locality and durability, instead of whether or not they provide requisite features. This paper presents the motivation and design of Syndicate, and gives the results of a preliminary evaluation showing that separating storage functionality from provider implementation is feasible in practice.

[1]  石田 三樹,et al.  Blackboard Learning System を用いた経済学講義の構築 , 2009 .

[2]  Lei Gao,et al.  PRACTI Replication for Large-Scale Systems , 2004 .

[3]  David G. Andersen,et al.  An Architecture for Internet Data Transfer , 2006, NSDI.

[4]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[5]  KyoungSoo Park,et al.  Scale and Performance in the CoBlitz Large-File Distribution Service , 2006, NSDI.

[6]  Predrag Buncic,et al.  CernVM-FS: delivering scientific software to globally distributed computing resources , 2011, NDM '11.

[7]  Larry L. Peterson,et al.  Experiences building PlanetLab , 2006, OSDI '06.

[8]  Vijay S. Pande,et al.  Folding@Home and Genome@Home: Using distributed computing to tackle previously intractable problem , 2009, 0901.0866.

[9]  Jeffrey I. Schiller,et al.  An Authentication Service for Open Network Systems. In , 1998 .

[10]  Robert Tappan Morris,et al.  Flexible, Wide-Area Storage for Distributed Systems with WheelFS , 2009, NSDI.

[11]  Eugene Ciurana,et al.  Google App Engine , 2009 .

[12]  Reagan Moore,et al.  iRODS Primer: Integrated Rule-Oriented Data System , 2010, iRODS Primer.

[13]  Nicholas Nye Wyant.  Google Documents: http://docs.google.com , 2012 .

[14]  Siddhartha Annapureddy,et al.  Shark: scaling file servers via cooperative caching , 2005, NSDI.

[15]  Drummond Reed,et al.  OpenID 2.0: a platform for user-centric identity management , 2006, DIM '06.

[16]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[17]  Divyakant Agrawal,et al.  G-Store: a scalable data store for transactional multi key access in the cloud , 2010, SoCC '10.

[18]  A. Rowstron,et al.  Towards predictable datacenter networks , 2011, SIGCOMM.

[19]  Miguel Castro,et al.  Farsite: federated, available, and reliable storage for an incompletely trusted environment , 2002, OPSR.

[20]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[21]  Dick Hardt,et al.  The OAuth 2.0 Authorization Framework , 2012, RFC.

[22]  Ali Ghodsi,et al.  Bolt-on causal consistency , 2013, SIGMOD '13.

[23]  Mohamed F. Mokbel,et al.  Deuteronomy: Transaction Support for Cloud Data , 2011, CIDR.

[24]  Antony I. T. Rowstron,et al.  IOFlow: a software-defined storage architecture , 2013, SOSP.

[25]  Ben Y. Zhao,et al.  Pond: The OceanStore Prototype , 2003, FAST.

[26]  Monica S. Lam,et al.  The collective: a cache-based system management architecture , 2005, NSDI.

[27]  Robert Tappan Morris,et al.  Ivy: a read/write peer-to-peer file system , 2002, OSDI '02.