Evaluation of a Performance Model of Lustre File System

As a large-scale global parallel file system, Lustre file system plays a key role in High Performance Computing (HPC) system, and the potential performance of such systems can be difficult to predict because the potential impact to application performance is not clearly understood. It is important to gain insights into the deliverable Lustre file system IO efficiency. In order to gain a good understanding on what and how to impact the performance of Lustre file system. This paper presents a study on performance evaluation of Lustre file systems and we propose a novel relative performance model to predict overhead under different performance factors. In our previous experiments, we discover that different performance factors have a closed correlation. In order to mining the correlations, we introduce relative performance model to predict performance differences between a pair of Lustre file system equipped with different performance factors. On average, relative model can predict bandwidth within 17%-28%. The results show our relative prediction model can obtain better prediction accuracy.

[1]  Arif Merchant,et al.  Issues and challenges in the performance analysis of real disk arrays , 2004, IEEE Transactions on Parallel and Distributed Systems.

[2]  Jeffrey S. Vetter,et al.  Performance characterization and optimization of parallel I/O on the Cray XT , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[3]  Zhu Jianjun.  Research on evaluations of several grey relational models adapt to grey relational axioms , 2009 .

[4]  Phillip M. Dickens,et al.  Towards an understanding of the performance of MPI-IO in Lustre file systems , 2008, 2008 IEEE International Conference on Cluster Computing.

[5]  Gregory R. Ganger,et al.  Modeling the relative fitness of storage , 2007, SIGMETRICS '07.

[6]  Jeffrey S. Vetter,et al.  Exploiting Lustre File Joining for Effective Collective IO , 2007, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[7]  Christos Faloutsos,et al.  Storage device performance prediction with CART models , 2004, The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004. (MASCOTS 2004). Proceedings..

[8]  Arif Merchant,et al.  A modular, analytical throughput model for modern disk arrays , 2001, MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[9]  Gregory R. Ganger,et al.  Relative fitness modeling , 2009, CACM.

[10]  Terence Kelly,et al.  Inducing Models of Black-Box Storage Arrays , 2004 .

[11]  Christos Faloutsos,et al.  Capturing the spatio-temporal behavior of real traffic data , 2002, Perform. Evaluation.

[12]  E. Anderson.  HPL – SSP – 2001 – 4 : Simple table-based modeling of storage devices , 2001 .

[13]  John Shalf,et al.  Using IOR to analyze the I/O Performance for HPC Platforms , 2007 .

[14]  Kees Wevers,et al.  Grey System Theory and Applications: A Way Forward , 2007 .

[15]  Jarek Nieplocha,et al.  Evaluation of active storage strategies for the lustre parallel file system , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[16]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[17]  Chang Liu,et al.  Comment on "Issues and Challenges in the Performance Analysis of Real Disk Arrays' , 2005, IEEE Trans. Parallel Distributed Syst..

[18]  Jeffrey S. Vetter,et al.  ParColl: Partitioned Collective I/O on the Cray XT , 2008, 2008 37th International Conference on Parallel Processing.

[19]  Depei Qian,et al.  A High Availability Mechanism for Parallel File System , 2005, APPT.

[20]  Qiang Cao,et al.  Approximate parameters analysis of a closed fork-join queue model in an object-based storage system , 2009, International Workshop on Information Data Storage and International Symposium on Optical Storage.

[21]  Christos Faloutsos,et al.  Data mining meets performance evaluation: fast algorithms for modeling bursty traffic , 2002, Proceedings 18th International Conference on Data Engineering.

[22]  John Wilkes,et al.  An introduction to disk drive modeling , 1994, Computer.

[23]  Jeffrey S. Vetter,et al.  Efficiency Evaluation of Cray XT Parallel IO Stack , 2007 .

[24]  Gregory R. Ganger,et al.  Self-* Storage: Brick-based Storage with Automated Administration (CMU-CS-03-178) , 2003 .

[25]  Jeffrey S. Vetter,et al.  Empirical Analysis of a Large-Scale Hierarchical Storage System , 2008, Euro-Par.

[26]  Yinliang Yue,et al.  High Availability Storage System Based on Two-Level Metadata Management , 2007, 2007 Japan-China Joint Workshop on Frontier of Computer Science and Technology (FCST 2007).