Coordinated process scheduling algorithms for coupled earth system models

It is becoming increasingly significant for humans to predict and understand future climate changes using coupled climate system models. Although the performance and scalability of individual physical components have improved over the past few years, coupled climate systems still suffer from low efficiency. This paper focuses on the process scheduling problem for the widely applied coupled earth system model (CESM). The proposed resource allocation strategies allow components to execute on a compromised suboptimal setup and still maintain approximately the best parallel speedup. With this flexible resource allocation strategy, we further propose a coordinated process scheduling algorithm (CPSA). More notably, we propose an upgraded version called CPSA‐B, which makes efficient resource sharing configurations, including resource allocation and process layout of components. We integrate CPSA and CPSA‐B as pre‐arrangement tools into the CESM program and deploy them on the Huawei Kunpeng platform. The speedup curves of the CESM components are prepared in advance, based on sampling tests. Experimental data show that CPSA‐B reduces up to 58% of the execution time compared with the CESM default strategy. The algorithm has low complexity and can efficiently find solutions for large input sizes.

[1]  Xiaohui Wei,et al.  MapReduce delay scheduling with deadline constraint , 2014, Concurr. Comput. Pract. Exp..

[2]  Mark A. Taylor,et al.  CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model , 2012, Int. J. High Perform. Comput. Appl..

[3]  Mariana Vertenstein,et al.  A new flexible coupler for earth system modeling developed for CCSM4 and CESM1 , 2012, Int. J. High Perform. Comput. Appl..

[4]  Mark A. Taylor,et al.  Performance of the Community Earth System Model , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[5]  Daniele Vigo,et al.  Bin packing approximation algorithms: Survey and classification , 2013 .

[6]  J. Randerson,et al.  Technical Description of version 4.0 of the Community Land Model (CLM) , 2010 .

[7]  Scott C. Doney,et al.  Evaluation of ocean model ventilation with CFC-11: comparison of 13 global ocean models , 2002 .

[8]  Weimin Zheng,et al.  An automatic performance model-based scheduling tool for coupled climate system models , 2019, J. Parallel Distributed Comput..

[9]  Yan Ma,et al.  Performance optimization and evaluation for parallel processing of big data in earth system models , 2019, Cluster Computing.

[10]  Arthur A. Mirin,et al.  A Scalable Implementation of a Finite-Volume Dynamical Core in the Community Atmosphere Model , 2005, Int. J. High Perform. Comput. Appl..

[11]  Chuan Wu,et al.  Optimus: an efficient dynamic resource scheduler for deep learning clusters , 2018, EuroSys.

[12]  Xiaohui Wei,et al.  An optimal checkpointing model with online OCI adjustment for stream processing applications , 2019, Concurr. Comput. Pract. Exp..

[13]  Bei Wang,et al.  An Empirical Study of HPC Workloads on Huawei Kunpeng 916 Processor , 2019, 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS).

[14]  W. Collins,et al.  Description of the NCAR Community Atmosphere Model (CAM 3.0) , 2004 .

[15]  Andrea Lodi,et al.  Two-dimensional packing problems: A survey , 2002, Eur. J. Oper. Res..

[16]  Leah Epstein,et al.  Tight results for Next Fit and Worst Fit with resource augmentation , 2010, Theor. Comput. Sci..

[17]  Darren J. Kerbyson,et al.  A Performance Model of the Parallel Ocean Program , 2005, Int. J. High Perform. Comput. Appl..

[18]  Xu Ji,et al.  CESMTuner: An Auto-tuning Framework for the Community Earth System Model , 2014, 2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS).

[19]  Wenguang Chen,et al.  Spread-n-share: improving application performance and cluster throughput with resource-aware job placement , 2019, SC.

[20]  Geppino Pucci,et al.  Universality in VLSI Computation , 2011, ParCo 2011.

[21]  Prasanna Balaprakash,et al.  Machine-Learning-Based Load Balancing for Community Ice Code Component in CESM , 2014, VECPAR.

[22]  Jay Walter Larson,et al.  Automatic Performance Prediction for Load-Balancing Coupled Models , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[23]  Sheri Mickelson,et al.  Community Earth System Model (CESM) , 2011, Encyclopedia of Parallel Computing.

[24]  Xiang Li,et al.  Integrated recovery and task allocation for stream processing , 2017, 2017 IEEE 36th International Performance Computing and Communications Conference (IPCCC).

[25]  Hiroshi Nagamochi,et al.  Exact algorithms for the two-dimensional strip packing problem with and without rotations , 2009, Eur. J. Oper. Res..

[26]  Shantenu Jha,et al.  Efficient Runtime Environment for Coupled Multi-physics Simulations: Dynamic Resource Allocation and Load-Balancing , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[27]  Patrick H. Worley,et al.  Performance Portability in the Physical Parameterizations of the Community Atmospheric Model , 2005, Int. J. High Perform. Comput. Appl..

[28]  John M. Levesque,et al.  Practical performance portability in the Parallel Ocean Program (POP) , 2005, Concurr. Pract. Exp..

[29]  Xiang Li,et al.  A Task Allocation Method for Stream Processing with Recovery Latency Constraint , 2018, Journal of Computer Science and Technology.

[30]  Hongliang Li,et al.  CPSA: A Coordinated Process Scheduling Algorithm for Coupled Earth System Model , 2020, 2020 29th International Conference on Computer Communications and Networks (ICCCN).

[31]  Henrik I. Christensen,et al.  Approximation and online algorithms for multidimensional bin packing: A survey , 2017, Comput. Sci. Rev..

[32]  David E. Dietrich,et al.  Application of a modified Arakawa ‘a’ grid ocean model having reduced numerical dispersion to the Gulf of Mexico circulation , 1998 .

[33]  Mariana Vertenstein,et al.  Computational performance of ultra-high-resolution capability in the Community Earth System Model , 2012, Int. J. High Perform. Comput. Appl..

[34]  Hiroshi Akima,et al.  A New Method of Interpolation and Smooth Curve Fitting Based on Local Procedures , 1970, JACM.