Parallization of Adaboost Algorithm through Hybrid MPI/OpenMP and Transactional Memory

This paper proposes a parallelization of the Adaboost algorithm through hybrid usage of MPI, OpenMP, and transactional memory. After detailed analysis of the Adaboost algorithm, we show that multiple levels of parallelism exists in the algorithm. We develop the lower level of parallelism through OpenMP and higher level parallelism through MPI. Software transactional memory are used to facilitate the management of shared data among different threads. We evaluated the Hybrid parallelized Adaboost algorithm on a heterogeneous PC cluster. And the result shows that nearly linear speedup can be achieved given a good load balancing scheme. Moreover, the hybrid parallelized Adaboost algorithm outperforms Purely MPI based approach by about 14% to 26%.

[1]  Håkan Grahn,et al.  Transactional memory , 2010, J. Parallel Distributed Comput..

[2]  Georg Hager,et al.  Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-Core SMP Nodes , 2009, 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing.

[3]  Stéphane Vialle,et al.  Implementation of the AdaBoost Algorithm for Large Scale Distributed Environments: Comparing JavaSpace and MPJ , 2009, 2009 15th International Conference on Parallel and Distributed Systems.

[4]  James R. Larus,et al.  Transactional Memory , 2006, Transactional Memory.

[5]  Nir Shavit,et al.  Transactional Locking II , 2006, DISC.

[6]  J. Chen,et al.  Performance measurements of scheduling strategies and parallel algorithms for a multiprocessor quick sort , 1984 .

[7]  Cesare Furlanello,et al.  Parallelizing AdaBoost by weights dynamics , 2007, Comput. Stat. Data Anal..

[8]  Erik Fridell,et al.  The mechanism for NOx storage , 2000 .

[9]  Zoran Obradovic,et al.  Boosting Algorithms for Parallel and Distributed Learning , 2022 .

[10]  It Informatics.  Haar-Like Features , 2009, Encyclopedia of Biometrics.

[11]  Zoran Obradovic,et al.  The distributed boosting algorithm , 2001, KDD '01.

[12]  Mark Bull,et al.  Development of mixed mode MPI / OpenMP applications , 2001, Sci. Program..

[13]  Daijin Kim,et al.  Robust Real-Time Face Detection Using Face Certainty Map , 2007, ICB.

[14]  J. Mark Bull,et al.  A Microbenchmark Suite for Mixed-Mode OpenMP/MPI , 2009, IWOMP.

[15]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[16]  Torvald Riegel,et al.  Dynamic performance tuning of word-based software transactional memory , 2008, PPoPP.

[17]  Fernando Lozano,et al.  Algorithms for parallel boosting , 2005, Fourth International Conference on Machine Learning and Applications (ICMLA'05).

[18]  Nir Shavit,et al.  Software transactional memory , 1995, PODC '95.

[19]  Bronis R. de Supinski,et al.  Evolving OpenMP in an Age of Extreme Parallelism, 5th International Workshop on OpenMP, IWOMP 2009, Dresden, Germany, June 3-5, 2009, Proceedings , 2009, IWOMP.