Cluster resource adjustment based on an improved artificial fish swarm algorithm in Mesos

At present, research of Mesos is in the early stage. Because of the two-level hierarchical scheduling, Mesos doesn't take the whole cluster into consideration in resource scheduling, which could cause load imbalance and low resource utilization. To solve above-mentioned problem, this paper proposed a resource adjustment plan based on an improved artificial fish swarm algorithm, it adopts a new behavior strategy. The users can adjust the mapping between salves and containers to improve cluster performance indexes without affecting the tasks' execution. The test results show the proposed resource adjustment achieved significantly increased load-balancing and resource utilization.

[1]  Randy H. Katz,et al.  Heterogeneity and dynamicity of clouds at scale: Google trace analysis , 2012, SoCC '12.

[2]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[3]  Benjamin Hindman,et al.  Dominant Resource Fairness: Fair Allocation of Multiple Resource Types , 2011, NSDI.

[4]  Jeffrey H. Meyerson,et al.  Ben Hindman on Apache Mesos , 2016, IEEE Softw..

[5]  Li Xiao-lei,et al.  Applications of artificial fish school algorithm in combinatorial optimization problems , 2004 .

[6]  Sarabjit Kaur,et al.  Cuckoo search approach for virtual machine consolidation in cloud data centre , 2015, International Conference on Computing, Communication & Automation.

[7]  John Holland,et al.  Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology , 1975 .

[8]  Carlo Curino,et al.  Apache Hadoop YARN: yet another resource negotiator , 2013, SoCC.

[9]  Randy H. Katz,et al.  Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center , 2011, NSDI.

[10]  Luiz André Barroso,et al.  Warehouse-Scale Computing: Entering the Teenage Decade , 2011, SIGARCH Comput. Archit. News.

[11]  Christina Delimitrou,et al.  Quasar: resource-efficient and QoS-aware cluster management , 2014, ASPLOS.

[12]  Mhand Hifi,et al.  A Reactive Local Search-Based Algorithm for the Multiple-Choice Multi-Dimensional Knapsack Problem , 2006, Comput. Optim. Appl..

[13]  Abhishek Verma,et al.  Large-scale cluster management at Google with Borg , 2015, EuroSys.

[14]  Chao Li,et al.  Fuxi: a Fault-Tolerant Resource Management and Job Scheduling System at Internet Scale , 2014, Proc. VLDB Endow..

[15]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[16]  Qing Liu,et al.  Application of an Artificial Fish Swarm Algorithm in Symbolic Regression , 2013, IEICE Trans. Inf. Syst..

[17]  Michael Abd-El-Malek,et al.  Omega: flexible, scalable schedulers for large compute clusters , 2013, EuroSys '13.

[18]  Xue Zhichun,et al.  Optimal operation of cascade reservoirs based on improved artificial fish swarm algorithm , 2011 .

[19]  Zhang Mei-feng,et al.  Hybrid Artificial Fish Swarm Optimization Algorithm Based on Mutation Operator and Simulated Annealing , 2006 .

[20]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[21]  Qing Liu,et al.  An Artificial Fish Swarm Algorithm for the Multicast Routing Problem , 2014, IEICE Trans. Commun..