Secure Dynamic Fragment and Replica Allocation in Large-Scale Distributed File Systems

We present a distributed algorithm for file allocation that guarantees high assurance, availability, and scalability in a large distributed file system. The algorithm can use replication and fragmentation schemes to allocate the files over multiple servers. The file confidentiality and integrity are preserved, even in the presence of a successful attack that compromises a subset of the file servers. The algorithm is adaptive in the sense that it changes the file allocation as the read-write patterns and the location of the clients in the network change. We formally prove that, assuming read-write patterns are stable, the algorithm converges toward an optimal file allocation, where optimality is defined as maximizing the file assurance.

[1]  Michael O. Rabin,et al.  Efficient dispersal of information for security, load balancing, and fault tolerance , 1989, JACM.

[2]  Pradeep K. Khosla,et al.  Selecting the Right Data Distribution Scheme for a Survivable Storage System (CMU-CS-01-120) , 2001 .

[3]  Pradeep K. Khosla,et al.  Survivable Information Storage Systems , 2000, Computer.

[4]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[5]  G. R. Blakley,et al.  Safeguarding cryptographic keys , 1899, 1979 International Workshop on Managing Requirements Knowledge (MARK).

[6]  Hugo Krawczyk,et al.  Secret Sharing Made Short , 1994, CRYPTO.

[7]  Mahadev Satyanarayanan,et al.  Andrew: a distributed personal computing environment , 1986, CACM.

[8]  Carl Smith,et al.  NFS Version 3: Design and Implementation , 1994, USENIX Summer.

[9]  Andrew V. Goldberg,et al.  Towards an archival Intermemory , 1998, Proceedings IEEE International Forum on Research and Technology Advances in Digital Libraries -ADL'98-.

[10]  A. Peressini,et al.  The Mathematics Of Nonlinear Programming , 1988 .

[11]  Marvin Theimer,et al.  Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs , 2000, SIGMETRICS '00.

[12]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[13]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[14]  Adi Shamir,et al.  How to share a secret , 1979, CACM.

[15]  David A. Goldberg,et al.  Design and Implementation of the Sun Network Filesystem , 1985, USENIX Conference Proceedings.

[16]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.