Adaptive Multi-copy Layout Algorithm Based on Mass Storage System

Large-scale storage systems face significant challenges in reliability and adaptability, thus it needs reliable, adaptive and effective data layout algorithms. Existing studies only partially meet these goals. This paper first puts forward a reliable copy data layout algorithm (RCDL) and an effective adaptive data layout algorithm (ADL), and on this basis, by combining the two algorithms, this paper proposes a multi-copy adaptive data layout algorithm MCADL, which can achieve better reliability, adaptability and effectiveness. The RCDL distributes the same copies to different storage devices to avoid the same replica on adjacent storage devices, thus obtaining higher redundancy and fault tolerance. The ADL algorithm combines the clustering algorithm with the consistent hash method, and introduces a small amount of virtual storage devices, greatly reducing the consumption of storage space. Data are distributed fairly according to the weights of the storage devices, so it is adaptive to system expansion and reduction. In order to utilize the respective advantages of RCDL and ADL, MCADL divides data into hot and cold data according to the data access frequency. RCDL layout is used for hot data and ADL layout is used for cold data. Theoretical and experimental results show that MCADL can obtain higher redundancy and fault tolerance and can fairly distribute data and add and remove adaptive storage devices according to the weights of storage devices, migrate optimal data amount when the scale of the storage system changes, and can quickly locate data, consuming less storage space.