Locality-aware fountain codes for massive distributed storage systems

Low repair locality of a distributed storage code has been shown to reduce strain on storage node input-output (I/O) resources during node repair operations after a failure. In this paper, we consider the use of Fountain codes for distributed storage systems and aim to understand the relationship between repair locality and code parameters for a systematic Fountain code. While the information-theoretic trade-off between repair locality and storage overhead has been understood and characterized, the challenge of choosing a locality value that satisfies multiple storage system design metrics is yet to be resolved. We approach this problem by deriving an expression for the probability distribution of repair locality in terms of the rateless code degree distribution coefficients and suggest that factoring this relationship into the code design process enables the design of rateless codes better adjusted to the needs of a massive distributed storage system.

[1]  Dimitris S. Papailiopoulos,et al.  XORing Elephants: Novel Erasure Codes for Big Data , 2013, Proc. VLDB Endow..

[2]  Kannan Ramchandran,et al.  A Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster , 2013, HotStorage.

[3]  Cheng Huang,et al.  On the Locality of Codeword Symbols , 2011, IEEE Transactions on Information Theory.

[4]  Subha Ramakrishna Gummadi,et al.  Coding and scheduling in networks for erasures and broadcast , 2011 .

[5]  Michael Luby,et al.  LT codes , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[6]  Yunnan Wu,et al.  A Survey on Network Codes for Distributed Storage , 2010, Proceedings of the IEEE.

[7]  Alexandros G. Dimakis,et al.  Repairable Fountain Codes , 2014, IEEE J. Sel. Areas Commun..

[8]  Joong Bum Rhim,et al.  Fountain Codes , 2010 .

[9]  Masoud Ardakani,et al.  An Efficient Binary Locally Repairable Code for Hadoop Distributed File System , 2014, IEEE Communications Letters.

[10]  Il-Min Kim,et al.  Binary Soliton-Like Rateless Coding for the Y-Network , 2011, IEEE Transactions on Communications.

[11]  Cheng Huang,et al.  Erasure Coding in Windows Azure Storage , 2012, USENIX Annual Technical Conference.