An Efficient Storage Technology for Object-Based Distributed Storage System

There have two effective ways to improve the overall efficiency of the object-based distributed storage system, one is network caching technology and the other is metadata server cluster load balancing strategy. Under the heterogeneous storage environments, object-based distributed file system face the key challenge of lacking caching mechanism for the high-frequency access files, metadata server load imbalance also reduce the overall efficiency. To address these challenges, we discuss the design and implementation of an object-based distributed file storage caching system, and propose a weight-based metadata server cluster load balancing strategy. It effectively improve the metadata processing capacity. Based on the results, it demonstrated the feasibility and the efficiency of our methods.

[1]  Scott A. Brandt,et al.  OBFS: A File System for Object-Based Storage Devices , 2004, MSST.

[2]  Daniel A. Menascé,et al.  QoS Issues in Web Services , 2002, IEEE Internet Comput..

[3]  Sandy Irani,et al.  Cost-Aware WWW Proxy Caching Algorithms , 1997, USENIX Symposium on Internet Technologies and Systems.

[4]  Ohad Rodeh,et al.  zFS - a scalable distributed file system using object disks , 2003, 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings..

[5]  Dharmendra S. Modha,et al.  SARC: Sequential Prefetching in Adaptive Replacement Cache , 2005, USENIX Annual Technical Conference, General Track.

[6]  Xiaoning Ding,et al.  DULO: an effective buffer cache management scheme to exploit both temporal and spatial locality , 2005, FAST'05.

[7]  Weili Xiong,et al.  A Robust Least Squares for Information Estimation Fusion , 2011 .

[8]  Kanishk Jain.  Object-based Storage , 2022 .

[9]  Gerhard Weikum,et al.  The LRU-K page replacement algorithm for database disk buffering , 1993, SIGMOD Conference.

[10]  Li Fan,et al.  Summary cache: a scalable wide-area web cache sharing protocol , 2000, TNET.

[11]  Gregory R. Ganger,et al.  Object-based storage , 2003, IEEE Commun. Mag..

[12]  Sean Quinlan,et al.  Venti: A New Approach to Archival Storage , 2002, FAST.

[13]  Tao Yang,et al.  The Panasas ActiveScale Storage Cluster - Delivering Scalable High Bandwidth Storage , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[14]  Carl Smith,et al.  NFS Version 3: Design and Implementation , 1994, USENIX Summer.

[15]  Nimrod Megiddo,et al.  ARC: A Self-Tuning, Low Overhead Replacement Cache , 2003, FAST.

[16]  Sang Lyul Min,et al.  LRFU: A Spectrum of Policies that Subsumes the Least Recently Used and Least Frequently Used Policies , 2001, IEEE Trans. Computers.

[17]  Carlos Maltzahn,et al.  Ceph: a scalable, high-performance distributed file system , 2006, OSDI '06.

[18]  Andrew J. Hutton,et al.  Lustre: Building a File System for 1,000-node Clusters , 2003 .