Priority Random Linear Codes in Distributed Storage Systems

Node churn and failures exist as fundamental characteristics in both peer-to-peer (P2P) and sensor networks. Peers in P2P networks are highly dynamic, whereas sensors are not dependable. As such, maintaining the persistence of periodically measured data in a scalable fashion has become a critical challenge in such systems, without the use of centralized servers. To better cope with node dynamics and failures, we propose priority random linear codes (RLCs), as well as their affiliated predistribution protocols, to maintain measurement data in different priorities, such that critical data have a higher opportunity to survive node failures than data of less importance. A salient feature of priority RLCs is the ability to partially recover more important subsets of the original data with higher priorities, when it is not feasible to recover all of them due to node dynamics. We present extensive analytical and experimental results to show the effectiveness of priority RLCs.

[1]  Baochun Li,et al.  Data Persistence in Large-Scale Sensor Networks with Decentralized Fountain Codes , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[2]  K. Jain,et al.  Practical Network Coding , 2003 .

[3]  R. Koetter,et al.  The benefits of coding over routing in a randomized setting , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[4]  Brad Karp,et al.  GPSR: greedy perimeter stateless routing for wireless networks , 2000, MobiCom '00.

[5]  K. Ramchandran,et al.  Random distributed multiresolution representations with significance querying , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[6]  Vinod M. Prabhakaran,et al.  Decentralized erasure codes for distributed networked storage , 2006, IEEE Transactions on Information Theory.

[7]  Qian Zhang,et al.  Partial Network Coding: Theory and Application for Continuous Sensor Data Collection , 2006, 200614th IEEE International Workshop on Quality of Service.

[8]  G. Blelloch Introduction to Data Compression * , 2022 .

[9]  Khalid Sayood,et al.  Introduction to Data Compression, Third Edition (Morgan Kaufmann Series in Multimedia Information and Systems) , 2005 .

[10]  Alexandros G. Dimakis,et al.  Network Coding for Distributed Storage Systems , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[11]  Baochun Li,et al.  How Practical is Network Coding? , 2006, 200614th IEEE International Workshop on Quality of Service.

[12]  Petri Kontkanen COMPUTING THE REGRET TABLE FOR MULTINOMIAL DATA , 2005 .

[13]  Rudolf Ahlswede,et al.  Network information flow , 2000, IEEE Trans. Inf. Theory.

[14]  Chuan Wu,et al.  Echelon: Peer-to-Peer Network Diagnosis with Network Coding , 2006, 200614th IEEE International Workshop on Quality of Service.

[15]  Madhu Sudan,et al.  Priority encoding transmission , 1996, IEEE Trans. Inf. Theory.

[16]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[17]  D. Lun,et al.  Methods for Efficient Network Coding , 2006 .

[18]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[19]  Muriel Medard,et al.  How good is random linear coding based distributed networked storage , 2005 .

[20]  Baochun Li,et al.  Differentiated Data Persistence with Priority Random Linear Codes , 2007, 27th International Conference on Distributed Computing Systems (ICDCS '07).

[21]  Donald F. Towsley,et al.  On optimal communication cost for gathering correlated data through wireless sensor networks , 2006, MobiCom '06.

[22]  Eli Upfal,et al.  Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2005 .

[23]  Muriel Médard,et al.  An algebraic approach to network coding , 2003, TNET.

[24]  Jon Feldman,et al.  Growth codes: maximizing sensor network data persistence , 2006, SIGCOMM.

[25]  Jeffrey Considine,et al.  Geometric generalizations of the power of two choices , 2004, SPAA '04.