Demonstrating Lustre over a 100Gbps wide area network of 3,500km

As part of the SCinet Research Sandbox at the Supercomputing 2011 conference, Indiana University (IU) demonstrated use of the Lustre high performance parallel file system over a dedicated 100 Gbps wide area network (WAN) spanning more than 3,500 km (2,175 mi). This demonstration functioned as a proof of concept and provided an opportunity to study Lustre's performance over a 100 Gbps WAN. To characterize the performance of the network and file system, low level iperf network tests, file system tests with the IOR benchmark, and a suite of real-world applications reading and writing to the file system were run over a latency of 50.5 ms. In this article we describe the configuration and constraints of the demonstration and outline key findings.

[1]  Miao Zhang,et al.  Driving Software Defined Networks with XSP , 2012, 2012 IEEE International Conference on Communications (ICC).

[2]  Matthias S. Müller,et al.  Performance and quality of service of data and video movement over a 100 Gbps testbed , 2013, Future Gener. Comput. Syst..

[3]  John Shalf,et al.  Using IOR to analyze the I/O Performance for HPC Platforms , 2007 .

[4]  Craig A. Stewart,et al.  Publications, presentations, and news pertaining to National Science Foundation grant number 0521433 – MRI: Acquisition of a High-Speed, High Capacity Storage System to Support Scientific Computing: The Data Capacitor , 2011 .

[5]  Scott Michael,et al.  A study of lustre networking over a 100 gigabit wide area network with 50 milliseconds of latency , 2012, DIDC '12.

[6]  G. Bryan,et al.  Introducing Enzo, an AMR Cosmology Application , 2004, astro-ph/0403044.

[7]  Yu Ma,et al.  Empowering distributed workflow with the data capacitor: maximizing lustre performance across the wide area network , 2007, SOCP '07.

[8]  Nancy Wilkins-Diehr,et al.  TeraGrid: Analysis of Organization, System Architecture, and Middleware Enabling New Types of Applications , 2006, High Performance Computing Workshop.

[9]  Daniel R. Harbeck,et al.  The WIYN One Degree Imager , 2008 .

[10]  Justin P. Miller,et al.  Enabling Lustre WAN for production use on the TeraGrid: a lightweight UID mapping scheme , 2010 .

[11]  Craig A. Stewart,et al.  Acquisition of a High-Speed, High Capacity Storage System to Support Scientific Computing: The Data Capacitor Final Report , 2009 .

[12]  Scott Michael,et al.  The Lustre File System and 100 Gigabit Wide Area Networking: An Example Case from SC11 , 2012, 2012 IEEE Seventh International Conference on Networking, Architecture, and Storage.

[13]  Scott Michael,et al.  A distributed workflow for an astrophysical OpenMP application: using the data capacitor over WAN to enhance productivity , 2010, HPDC '10.

[14]  Stephen C. Simms,et al.  Wide Area Filesystem Performance using Lustre on the TeraGrid , 2007 .

[15]  M. Bazhenov,et al.  Ionic Dynamics Mediate Spontaneous Termination of Seizures and Postictal Depression State , 2011, The Journal of Neuroscience.

[16]  Roger Smith,et al.  A compelling case for a centralized filesystem on the TeraGrid: enhancing an astrophysical workflow with the data capacitor WAN as a test case , 2010 .

[17]  Westone,et al.  Home Page , 2004, 2022 2nd International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA).

[18]  Guido Juckeland,et al.  Comprehensive Performance Tracking with Vampir 7 , 2009, Parallel Tools Workshop.

[19]  David Medvigy,et al.  The Ocean–Land–Atmosphere Model: Optimization and Evaluation of Simulated Radiative Fluxes and Precipitation , 2010 .