Understanding the Effect of Multiple Factors on a Parallel File System's Performance

This research work presents an investigation of the impact of a wide range of factors on the performance of parallel file systems (PFSs). It is the result of an extensive test campaign with three distinct computing platforms and value variations for eleven factors that advance the understanding of PFSs' behaviour under different conditions. Our main contributions are the characterization of effects not fully explained or misunderstood in the literature previously. First, we demonstrate that no significant performance variation (≈ 6%) is observed when choosing one among four TCP congestion-avoidance algorithms. Second, we detail the effect of the page cache of I/O nodes on a PFS's write throughput and how it relates to other factors.

[1]  Amy W. Apon,et al.  Parallel file system measurement and modeling using colored petri nets , 2012, ICPE '12.

[2]  Frank B. Schmuck,et al.  GPFS: A Shared-Disk File System for Large Computing Clusters , 2002, FAST.

[3]  Robert Ross,et al.  Implementation and performance of a parallel file system for high performance distributed applications , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[4]  Robert B. Ross,et al.  On the duality of data-intensive file system design: Reconciling HDFS and PVFS , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[5]  Mario A. R. Dantas,et al.  A survey into performance and energy efficiency in HPC, cloud and big data environments , 2014, Int. J. Netw. Virtual Organisations.

[6]  Peter J. Braam,et al.  Lustre: The intergalactic file system , 2002 .

[7]  Rupak Biswas,et al.  I/O performance characterization of Lustre and NASA applications on Pleiades , 2012, 2012 19th International Conference on High Performance Computing.

[8]  Michael Anthony Bauer,et al.  A Data Management in a Private Cloud Storage Environment Utilizing High Performance Distributed File Systems , 2013, 2013 Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises.