Efficient Content Distribution and Storage P2P System based on Information Dispersal

Recent advances in communication networks and the computing capabilities of computers have opened up new opportunities for content storage and distribution, where cloud storage and media distribution over the Internet have gained significant popularity. However, most computer systems may have failures on both the software and hardware side. Furthermore, as the Internet has grown, traditional server based system have introduced new problems such as bandwidth bottlenecks, limited performance and dependability. In this paper, we propose a fault-tolerant collaborative architecture for storing and distributing multimedia files based on the information dispersal algorithm (IDA). Our proposed architecture is based on the P2P paradigm in such a way that the nodes can take both the server and client roles at the same time. In this way, a file can be allocated efficiently to many peers by using the information dispersal algorithm, and only a subset of these peers are required to rebuild the original file.

[1]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[2]  Francisco de Asís López-Fuentes,et al.  Multi-source video multicast in peer-to-peer networks , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[3]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[4]  Michael O. Rabin,et al.  Efficient dispersal of information for security, load balancing, and fault tolerance , 1989, JACM.

[5]  Jacob R. Lorch,et al.  Farsite: federated, available, and reliable storage for an incompletely trusted environment , 2002, OSDI '02.

[6]  Anne-Marie Kermarrec,et al.  Pulp: An adaptive gossip-based dissemination protocol for multi-source message streams , 2012, Peer Peer Netw. Appl..

[7]  Florian Schintke,et al.  Peer-to-Peer Computing , 2010, Euro-Par.

[8]  A. De Santis,et al.  On information dispersal algorithms , 2002, Proceedings IEEE International Symposium on Information Theory,.

[9]  Andrew V. Goldberg,et al.  A prototype implementation of archival Intermemory , 1999, DL '99.

[10]  Jesús Carretero,et al.  Fault-tolerant middleware based on multistream pipeline for private storage services , 2012, 2012 International Conference for Internet Technology and Secured Transactions.

[11]  Wilson C. Hsieh,et al.  Bigtable: A Distributed Storage System for Structured Data , 2006, TOCS.

[12]  Müge Sayit,et al.  Adaptive, incentive and scalable dynamic tree overlay for P2P live video streaming , 2016, Peer-to-Peer Netw. Appl..

[13]  Chen-Hua Shih,et al.  Advanced bootstrap and adjusted bandwidth for content distribution in peer-to-peer live streaming , 2015, Peer Peer Netw. Appl..

[14]  Azer Bestavros An Adaptive Information Dispersal Algorithm for Time-Critical Reliable Communication , 1994 .

[15]  John Gantz,et al.  The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East , 2012 .

[16]  Wei-Ho Chung,et al.  An Efficient $(n,k)$ Information Dispersal Algorithm Based on Fermat Number Transforms , 2013, IEEE Transactions on Information Forensics and Security.

[17]  Himabindu Pucha,et al.  Exploiting Similarity for Multi-Source Downloads Using File Handprints , 2007, NSDI.