Performance Analysis of MapReduce Program in Heterogeneous Cloud Computing

The research of Hadoop is an important part of cloud computing industry, and Hadoop performance research is a key research direction. The Hadoop performance analysis as a basic work can provide important reference for other performance optimization researches. In this paper, based on previous researches of server performance analysis, we propose a node performance measurement method on Hadoop. We describe in detail how to measure the performance value of each node in heterogeneous Hadoop cluster and evaluate measurement results by running MapReduce programs. Meanwhile, the method has also been implemented and evaluated in real-world Hadoop cluster. Experiment results show that the method can accurately measure the performance value of each node. Based on this research, users can have a comprehensive and objective understanding of their own Hadoop cluster and then make optimization and improvement on Hadoop

[1]  Robert L. Grossman,et al.  Ieee Transactions on Parallel and Distributed Systems, Manuscript Id towards Efficient and Simplified Distributed Data Intensive Computing* , 2022 .

[2]  Shigen Shen,et al.  Task Scheduling Optimization in Cloud Computing Based on Heuristic Algorithm , 2012, J. Networks.

[3]  Jayanti Vemulapati,et al.  Demystifying Cloud Benchmarking Paradigm - An in Depth View , 2012, 2012 IEEE 36th Annual Computer Software and Applications Conference.

[4]  Yunqi Lei,et al.  Geometric Features of 3D Face and Recognition of It by PCA , 2011, J. Multim..

[5]  Carl Staelin.  lmbench: an extensible micro‐benchmark suite , 2005, Softw. Pract. Exp..

[6]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[7]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[8]  Lavanya Ramakrishnan,et al.  Benchmarking MapReduce Implementations for Application Usage Scenarios , 2011, 2011 IEEE/ACM 12th International Conference on Grid Computing.

[9]  Xinli Wang,et al.  Cloud computing performance benchmarking and virtual machine launch time , 2012, SIGITE '12.

[10]  A. Akila,et al.  THE SURVEY ON MAPREDUCE , 2012 .

[11]  Juebo Wu,et al.  A Cloud Model-based Approach for Facial Expression Synthesis , 2011, J. Multim..

[12]  Randy H. Katz,et al.  Improving MapReduce Performance in Heterogeneous Environments , 2008, OSDI.

[13]  Nicholas Hatt.  Benchmarking Operating Systems , 2008 .

[14]  Sugang Ma,et al.  A Review on Cloud Computing Development , 2012, J. Networks.

[15]  C. Murray Woodside,et al.  Performance analysis of distributed server systems , 2000 .

[16]  Kyuseok Shim,et al.  MapReduce Algorithms for Big Data Analysis , 2012, Proc. VLDB Endow..