Cluster-based implementation of resource brokering strategy for parallel training of neural networks

The implementation issues of a cluster-based resource brokering strategy intended for efficient parallelization of neural networks training are presented in this paper. We describe a strategy of resource brokering based on the prediction of execution time and parallelization efficiency of algorithms using a BSP computation model and Pareto optimality with a weight coefficients approach for choosing optimal solution. Our results show a reasonable adaptation of the resource brokering strategy to the environment of a real computational cluster providing the minimal total time to delivery of the parallel application.

[1]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[2]  Volodymyr Turchenko,et al.  Efficiency Analysis of Parallel Batch Pattern NN Training Algorithm on General-Purpose Supercomputer , 2009, IWANN.

[3]  Rob H. Bisseling,et al.  Parallel scientific computation - a structured approach using BSP and MPI , 2004 .

[4]  Fatos Xhafa,et al.  Computational models and heuristic methods for Grid scheduling problems , 2010, Future Gener. Comput. Syst..

[5]  Radu Prodan,et al.  Towards a general model of the multi-criteria workflow scheduling on the grid , 2009, Future Gener. Comput. Syst..

[6]  Rob H. Bisseling,et al.  Parallel Scientific Computation , 2004 .

[7]  George Bosilca,et al.  Improvement of parallelization efficiency of batch pattern BP training algorithm using Open MPI , 2010, International Conference on Conceptual Structures.

[8]  Volodymyr Turchenko,et al.  Strategy of resource brokering for efficient parallelization of MLP training , 2010, 2010 International Conference on High Performance Computing & Simulation.

[9]  Johan Tordsson,et al.  Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions , 2008, Future Gener. Comput. Syst..

[10]  Rajkumar Buyya,et al.  Time and cost trade-off management for scheduling parallel applications on Utility Grids , 2010, Future Gener. Comput. Syst..

[11]  Liana L. Fong,et al.  Grid broker selection strategies using aggregated resource information , 2010, Future Gener. Comput. Syst..

[12]  David B. Skillicorn,et al.  Using the BSP cost model to optimise parallel neural network training , 1998, Future Gener. Comput. Syst..

[13]  Volodymyr Turchenko,et al.  Estimation of Computational Complexity of Sensor Accuracy Improvement Algorithm Based on Neural Networks , 2001, ICANN.

[14]  Lucio Grandinetti,et al.  Parallel batch pattern BP training algorithm of recurrent neural network , 2010, 2010 IEEE 14th International Conference on Intelligent Engineering Systems.

[15]  Jarek Nabrzyski,et al.  A multicriteria approach to two-level hierarchy scheduling in grids , 2008, J. Sched..

[16]  Volodymyr Turchenko,et al.  Application of BSP-Based Computational Cost Model to Predict Parallelization Efficiency of MLP Training Algorithm , 2010, ICANN.