Improved load balancing for distributed file system using self acting and adaptive loading data migration process

Load balancing in a distributed file system is becoming essential as a distributed file system is gaining more popularity both in use and research. Resource utilization is very much important as it affects the cost of the system, response time, performance etc. Now a days dynamic load balancer is adopted by most of distributed file systems. Adaptive loading data migration and Self acting load balancing proves better performance, which behaves centralized and distributed way respectively. A Modified approach to create new distributed, dynamic load balancer for distributed file system is adopted. In which, the algorithm considers constraints as network load, disk IO load, Disk capacity, load which was not considered in earlier approaches. The load balancing problem is solved based on constraint satisfaction theory, which sets the optimum level so that system throughput, response time is improved. Experimental results show that overall performance is improved by 8 to 9%.

[1]  Gerhard Weikum,et al.  Data partitioning and load balancing in parallel disk systems , 1998, The VLDB Journal.

[2]  Evgenia Smirni,et al.  Workload Characterization of Input/Output Intensive Parallel Applications , 1997, Computer Performance Evaluation.

[3]  Gong Zhang,et al.  Adaptive Data Migration in Multi-tiered Storage Based Cloud Environment , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[4]  Ali Hamzeh,et al.  A new multi-agent algorithm for solving constraint satisfaction problems , 2013, The 5th Conference on Information and Knowledge Technology.

[5]  Azer Bestavros,et al.  Speculative data dissemination and service to reduce server load, network traffic and service time in distributed information systems , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[6]  Sandeep Uttamchandani,et al.  Risk-Modulated Proactive Data Migration for Maximizing Storage System Utility , 2006 .

[7]  Randal C. Burns,et al.  CA-NFS: A congestion-aware network file system , 2009, TOS.

[8]  Yao Sun,et al.  A file assignment strategy independent of workload characteristic assumptions , 2009, TOS.

[9]  Margo I. Seltzer,et al.  Passive NFS Tracing of Email and Research Workloads , 2003, FAST.

[10]  Thomas E. Anderson,et al.  A Comparison of File System Workloads , 2000, USENIX Annual Technical Conference, General Track.

[11]  Olivia R. Liu Sheng.  Dynamic file migration in distributed computer systems , 1990, CACM.

[12]  Peter Scheuermann,et al.  File Assignment in Parallel I/O Systems with Minimal Variance of Service Time , 2000, IEEE Trans. Computers.

[13]  Limin Xiao,et al.  A dynamic and adaptive load balancing strategy for parallel file system with large-scale I/O servers , 2012, J. Parallel Distributed Comput..

[14]  Dan Feng,et al.  ALDM: Adaptive Loading Data Migration in Distributed File Systems , 2013, IEEE Transactions on Magnetics.

[15]  Yang Yu,et al.  A Balanced Allocation Strategy for File Assignment in Parallel I/O Systems , 2010, 2010 IEEE Fifth International Conference on Networking, Architecture, and Storage.

[16]  Lawrence W. Dowdy,et al.  Comparative Models of the File Assignment Problem , 1982, CSUR.

[17]  John Shalf,et al.  The International Exascale Software Project roadmap , 2011, Int. J. High Perform. Comput. Appl..

[18]  Yiming Hu,et al.  Efficient, proximity-aware load balancing for DHT-based P2P systems , 2005, IEEE Transactions on Parallel and Distributed Systems.

[19]  Michele Colajanni,et al.  Models and framework for supporting runtime decisions in Web-based systems , 2008, TWEB.

[20]  Sudarshan S. Deshmukh,et al.  A Survey: Load Balancing for Distributed File System , 2015 .

[21]  Mary Baker,et al.  Measurements of a distributed file system , 1991, SOSP '91.