Strategy of resource brokering for efficient parallelization of MLP training

A strategy of resource brokering for efficient parallelization of the parallel batch pattern back propagation training algorithm of a multilayer perceptron is presented in this paper. A BSP-based computational cost model of the parallel algorithm is used for the prediction of its execution time and parallelization efficiency. The strategy of resource brokering is based on Pareto optimality with the weighted sum approach for choosing optimal solutions for efficient parallelization of the algorithm. The results of experimental research show that the developed resource brokering strategy has good conformity with the desired scheduling policy of minimization of the execution time of the algorithm with maximization of the parallelization efficiency in the most economic way.

[1]  José Luis Bosque,et al.  Study of neural net training methods in parallel and distributed architectures , 2010, Future Gener. Comput. Syst..

[2]  Jonathan M. D. Hill,et al.  Theory, Practice, and a Tool for BSP Performance Prediction , 1996, Euro-Par, Vol. II.

[3]  Lucio Grandinetti,et al.  Investigation of computational cost model of MLP parallel batch training algorithm , 2009, 2009 IEEE Symposium on Industrial Electronics & Applications.

[4]  Rajkumar Buyya,et al.  Scheduling Parallel Applications on Utility Grids: Time and Cost Trade-off Management , 2009, ACSC.

[5]  Liana L. Fong,et al.  Grid broker selection strategies using aggregated resource information , 2010, Future Gener. Comput. Syst..

[6]  David B. Skillicorn,et al.  Using the BSP cost model to optimise parallel neural network training , 1998, Future Gener. Comput. Syst..

[7]  Pablo A. Estévez,et al.  A scalable parallel algorithm for training a hierarchical mixture of neural experts , 2002, Parallel Comput..

[8]  Fatos Xhafa,et al.  Computational models and heuristic methods for Grid scheduling problems , 2010, Future Gener. Comput. Syst..

[9]  Radu Prodan,et al.  Towards a general model of the multi-criteria workflow scheduling on the grid , 2009, Future Gener. Comput. Syst..

[10]  Volodymyr Turchenko,et al.  Efficiency Analysis of Parallel Batch Pattern NN Training Algorithm on General-Purpose Supercomputer , 2009, IWANN.

[11]  G. Fagg,et al.  Flexible collective communication tuning architecture applied to Open MPI , 2006 .

[12]  Li-Chiu Chang,et al.  An efficient parallel algorithm for LISSOM neural network , 2002, Parallel Comput..

[13]  Volodymyr Turchenko,et al.  Estimation of Computational Complexity of Sensor Accuracy Improvement Algorithm Based on Neural Networks , 2001, ICANN.

[14]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[15]  Jarek Nabrzyski,et al.  A multicriteria approach to two-level hierarchy scheduling in grids , 2008, J. Sched..

[16]  Rob H. Bisseling,et al.  Parallel scientific computation - a structured approach using BSP and MPI , 2004 .

[17]  Emmanouel A. Varvarigos,et al.  A comparison of centralized and distributed meta-scheduling architectures for computation and communication tasks in Grid networks , 2009, Comput. Commun..

[18]  Rob H. Bisseling,et al.  Parallel Scientific Computation , 2004 .

[19]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[20]  George Bosilca,et al.  Improvement of parallelization efficiency of batch pattern BP training algorithm using Open MPI , 2010, International Conference on Conceptual Structures.

[21]  Johan Tordsson,et al.  Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions , 2008, Future Gener. Comput. Syst..

[22]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[23]  I. Mazin,et al.  Theory , 1934 .

[24]  Lucio Grandinetti,et al.  Efficiency research of batch and single pattern MLP parallel training algorithms , 2009, 2009 IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications.