A File Transfer Service with Client/Server, P2P and Wide Area Storage Protocols

The last years have seen the emergence of new P2P file distribution protocols such as BitTorrent as well as new Wide Area Storage based on Web Service technology. In this paper, we propose a multi-protocol file transfer service which supports client/server and P2P protocols, as well as Wide Area Storage such as Amazon S3. We describe the mechanisms used to ensure file transfer monitoring and reliability. We explain how to plug-in new or existing protocols and we give evidence of the versatility of the framework by implementing the HTTP, FTP and BitTorrent protocols and access to the Amazon S3 and IBP Wide Area Storage. Finally, we report on basic performance evaluation of our framework, both in a Grid context and on the Internet.

[1]  Gilles Fedak,et al.  BitDew: a programmable environment for large-scale data management and distribution , 2008, HiPC 2008.

[2]  Rayadurgam Srikant,et al.  Modeling and performance analysis of BitTorrent-like peer-to-peer networks , 2004, SIGCOMM 2004.

[3]  Franck Cappello,et al.  Grid'5000: A Large Scale And Highly Reconfigurable Experimental Grid Testbed , 2006, Int. J. High Perform. Comput. Appl..

[4]  Christos Gkantsidis,et al.  Network coding for large scale content distribution , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[5]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[6]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[7]  Dahlia Malkhi,et al.  On collaborative content distribution using multi-message gossip , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[8]  Alessandro Bassi,et al.  The Internet Backplane Protocol: A Study in Resource Sharing , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[9]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[10]  Matei Ripeanu,et al.  Amazon S3 for science grids: a viable solution? , 2008, DADC '08.

[11]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[12]  Simson L. Garfinkel,et al.  An Evaluation of Amazon's Grid Computing Services: EC2, S3, and SQS , 2007 .

[13]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[14]  Mikel Izal,et al.  Dissecting BitTorrent: Five Months in a Torrent's Lifetime , 2004, PAM.