The Design and Implementation of Multi-Cloud Based Distributed Storage Platform with Random Linear Coding

Nowadays, distributed storage systems have been widely used in the world, e.g, Google Drive, OneDrive, DropBox, Baidu PCS, etc.. Although these cloud storage services significantly facilitate the file storage for users, the performance of them varies widely when using in different countries. For example, Google Drive has high upload/download speed in America while it has very low upload/download speed in China. Therefore, to enhance the reliability and security of the cloud storage, in this paper, we firstly design a multi-cloud based distributed storage platform, named as, Normal Drive, by utilizing random linear coding and APIs provided by multiple cloud disks. We then implement the Normal Drive and conduct extensive evaluation from different aspects in five different locations around the world to test the performance of it. The experimental results show that the proposed Normal Drive can significantly improve the transmission performance and enhance the reliability of file storage when comparing with single-cloud based distributed storage system.

[1]  Frank B. Schmuck,et al.  GPFS: A Shared-Disk File System for Large Computing Clusters , 2002, FAST.

[2]  Jin Wang,et al.  Optimal Linear Network Coding Design for Secure Unicast with Multiple Streams , 2010, 2010 Proceedings IEEE INFOCOM.

[3]  Baochun Li,et al.  Random Network Coding in Peer-to-Peer Networks: From Theory to Practice , 2011, Proceedings of the IEEE.

[4]  Rudolf Ahlswede,et al.  Network information flow , 2000, IEEE Trans. Inf. Theory.

[5]  Andrea C. Arpaci-Dusseau,et al.  ViewBox: integrating local file systems with cloud storage services , 2014, FAST.

[6]  Jin Wang,et al.  An Efficient Reliable Communication Scheme in Wireless Sensor Networks Using Linear Network Coding , 2012, Int. J. Distributed Sens. Networks.

[7]  Bin Zhou,et al.  Scalable Performance of the Panasas Parallel File System , 2008, FAST.

[8]  Muriel Médard,et al.  Random Linear Network Coding: A free cipher? , 2007, 2007 IEEE International Symposium on Information Theory.

[9]  Fangming Liu,et al.  UniDrive: Synergize Multiple Consumer Cloud Storage Services , 2015, Middleware.

[10]  Jin Wang,et al.  Modeling and Optimal Design of Linear Network Coding for Secure Unicast with Multiple Streams , 2013, IEEE Transactions on Parallel and Distributed Systems.

[11]  Jin Wang,et al.  The Design and Implementation of Random Linear Network Coding Based Distributed Storage System in Dynamic Networks , 2018, ICA3PP.

[12]  Bo Li,et al.  Cinematic-Quality VoD in a P2P Storage Cloud: Design, Implementation and Measurements , 2013, IEEE Journal on Selected Areas in Communications.

[13]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[14]  Carlos Maltzahn,et al.  Ceph: a scalable, high-performance distributed file system , 2006, OSDI '06.

[15]  Marten van Dijk,et al.  Iris: a scalable cloud file system with efficient integrity checks , 2012, ACSAC '12.

[16]  Tracey Ho,et al.  A Random Linear Network Coding Approach to Multicast , 2006, IEEE Transactions on Information Theory.

[17]  David E. Culler,et al.  PlanetLab: an overlay testbed for broad-coverage services , 2003, CCRV.