Towards the design of optimal data redundancy schemes for heterogeneous cloud storage infrastructures

Nowadays, data storage requirements from end-users are growing, demanding more capacity, more reliability and the capability to access information from anywhere. Cloud storage services meet this demand by providing transparent and reliable storage solutions. Most of these solutions are built on distributed infrastructures that rely on data redundancy to guarantee a 100% of data availability. Unfortunately, existing redundancy schemes very often assume that resources are homogeneous, an assumption that may increase storage costs in heterogeneous infrastructures - e.g., clouds built of voluntary resources. In this work, we analyze how distributed redundancy schemes can be optimally deployed over heterogeneous infrastructures. Specifically, we are interested in infrastructures where nodes present different online availabilities. Considering these heterogeneities, we present a mechanism to measure data availability more precisely than existing works. Using this mechanism, we infer the optimal data placement policy that reduces the redundancy used, and then its associated overheads. In heterogeneous settings, our results show that data redundancy can be reduced up to 70%.

[1]  Karl Aberer,et al.  Internet-Scale Storage Systems under Churn -- A Study of the Steady-State using Markov Models , 2006, Sixth IEEE International Conference on Peer-to-Peer Computing (P2P'06).

[2]  D. M. Chiu,et al.  Erasure code replication revisited , 2004, Proceedings. Fourth International Conference on Peer-to-Peer Computing, 2004. Proceedings..

[3]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[4]  Ravi Jain,et al.  An Experimental Study of the Skype Peer-to-Peer VoIP System , 2005, IPTPS.

[5]  Taoufik En-Najjary,et al.  Proactive replication in distributed storage systems using machine availability estimation , 2007, CoNEXT '07.

[6]  Kian-Lee Tan,et al.  PeerStore: better performance by relaxing in peer-to-peer backup , 2004, Proceedings. Fourth International Conference on Peer-to-Peer Computing, 2004. Proceedings..

[7]  Dejan S. Milojicic,et al.  Open Cirrus TM cloud computing testbed: federated data centers for open source systems and services research , 2009, CloudCom 2009.

[8]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[9]  John Kubiatowicz,et al.  Erasure Coding Vs. Replication: A Quantitative Comparison , 2002, IPTPS.

[10]  Brian D. Noble,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Pastiche: Making Backup Cheap and Easy , 2022 .

[11]  Guihai Chen,et al.  Redundancy Schemes for High Availability in DHTs , 2005, ISPA.

[12]  Ernst W. Biersack,et al.  Hierarchical Codes: How to Make Erasure Codes Attractive for Peer-to-Peer Storage Systems , 2008, 2008 Eighth International Conference on Peer-to-Peer Computing.

[13]  Marvin Theimer,et al.  Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs , 2000, SIGMETRICS '00.

[14]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[15]  Yuhui Deng,et al.  A heterogeneous storage grid enabled by grid service , 2007, OPSR.

[16]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[17]  Stefan Savage,et al.  Total Recall: System Support for Automated Availability Management , 2004, NSDI.

[18]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[19]  Rodrigo Rodrigues,et al.  High Availability in DHTs: Erasure Coding vs. Replication , 2005, IPTPS.

[20]  Jean-Marc Vincent,et al.  Mining for statistical models of availability in large-scale distributed systems: An empirical study of SETI@home , 2009, 2009 IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems.

[21]  Kian-Lee Tan,et al.  PeerStore: better performance by relaxing in peer-to-peer backup , 2004 .

[22]  Andreas Haeberlen,et al.  Proactive Replication for Data Durability , 2006, IPTPS.

[23]  Reagan Moore,et al.  Data grids, collections, and grid bricks , 2003, 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings..

[24]  Abhishek Chandra,et al.  Nebulas: Using Distributed Voluntary Resources to Build Clouds , 2009, HotCloud.

[25]  Donald E. Knuth,et al.  The art of computer programming, volume 3: (2nd ed.) sorting and searching , 1998 .

[26]  Zheng Zhang,et al.  Reperasure: replication protocol using erasure-code in peer-to-peer storage network , 2002, 21st IEEE Symposium on Reliable Distributed Systems, 2002. Proceedings..

[27]  Marc Sánchez Artigas,et al.  Heterogeneity-Aware Erasure Codes for Peer-to-Peer Storage Systems , 2009, 2009 International Conference on Parallel Processing.

[28]  Alexandros G. Dimakis,et al.  Network Coding for Distributed Storage Systems , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[29]  Andreas Haeberlen,et al.  Efficient Replica Maintenance for Distributed Storage Systems , 2006, NSDI.

[30]  Rodrigo Rodrigues,et al.  Proceedings of Hotos Ix: the 9th Workshop on Hot Topics in Operating Systems Hotos Ix: the 9th Workshop on Hot Topics in Operating Systems High Availability, Scalable Storage, Dynamic Peer Networks: Pick Two , 2022 .

[31]  Kenneth Ward Church,et al.  On Delivering Embarrassingly Distributed Cloud Services , 2008, HotNets.

[32]  Chris Rose,et al.  A Break in the Clouds: Towards a Cloud Definition , 2011 .