A generalized replica placement strategy to optimize latency in a wide area distributed storage system

Wide area distributed storage systems leverage the scale of the Internet to provide vast amounts of data storage capabilities. The typical building block is a peer-to-peer system that can pool disparate distributed resources into a cohesive storage system. However, the participant nodes are relatively unreliable and thus, distributed storage systems must take care to provide replicas of data objects. A common problem is how to allocate objects to nodes in the system and previous work has allocated objects based on popularity. However, these schemes are not general enough to minimize latency or deal with redundancy schemes such as erasure codes which are based on replication of object fragments. In this paper, we present a strategy for placement of replica fragments particularly with respect to the minimization of data communication latencies.

[1]  Jussi Kangasharju,et al.  Internet content distribution , 2002 .

[2]  Ethan L. Miller,et al.  POTSHARDS: Secure Long-Term Storage Without Encryption , 2007, USENIX Annual Technical Conference.

[3]  Lili Qiu,et al.  On the placement of Web server replicas , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[4]  Michal Szymaniak,et al.  Latency-Driven Replica Placement , 2006 .

[5]  Antony I. T. Rowstron,et al.  PAST: a large-scale, persistent peer-to-peer storage utility , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[6]  Yannis Manolopoulos,et al.  A latency-based object placement approach in content distribution networks , 2005, Third Latin American Web Congress (LA-WEB'2005).

[7]  Robert Tappan Morris,et al.  Ivy: a read/write peer-to-peer file system , 2002, OSDI '02.

[8]  Efficient Reliable Internet Storage ∗ , 2004 .

[9]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[10]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[11]  John Kubiatowicz,et al.  Erasure Coding Vs. Replication: A Quantitative Comparison , 2002, IPTPS.

[12]  Sugih Jamin,et al.  Inet-3.0: Internet Topology Generator , 2002 .

[13]  Marvin Theimer,et al.  Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs , 2000, SIGMETRICS '00.

[14]  Magnus Karlsson,et al.  Taming aggressive replication in the Pangaea wide-area file system , 2002, OPSR.

[15]  Roger Wattenhofer,et al.  Competitive Hill-Climbing Strategies for Replica Placement in a Distributed File System , 2001, DISC.

[16]  Ben Y. Zhao,et al.  Awarded Best Student Paper! - Pond: The OceanStore Prototype , 2003 .

[17]  Ben Y. Zhao,et al.  Pond: The OceanStore Prototype , 2003, FAST.