Enabling Immediate Access to Earth Science Models through Cloud Computing: Application to the GEOS-Chem Model

AbstractCloud computing platforms can provide fast and easy access to complex Earth science models and large datasets. This article presents a mature capability for running the GEOS-Chem global 3D ...

[1]  Preston M. Smith,et al.  Cost-Effective HPC: The Community or the Cloud? , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[2]  D. Jacob,et al.  Global modeling of tropospheric chemistry with assimilated meteorology : Model description and evaluation , 2001 .

[3]  Zhenlong Li,et al.  Building Model as a Service to support geosciences , 2017, Comput. Environ. Urban Syst..

[4]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[5]  T. Langkamp.  Interactive comment on “Influence of the compiler on multi-CPU performance of WRFv3” by T. Langkamp , 2011 .

[6]  Xiuhong Chen,et al.  Running climate model on a commercial cloud computing environment: A case study using Community Earth System Model (CESM) on Amazon AWS , 2017, Comput. Geosci..

[7]  François Chollet,et al.  Deep Learning with Python , 2017 .

[8]  D. Jacobsen,et al.  Contain This, Unleashing Docker for HPC , 2015 .

[9]  D. Weisenstein,et al.  Development and evaluation of the unified tropospheric–stratospheric chemistry extension (UCX) for the global chemistry-transport model GEOS-Chem , 2014 .

[10]  D. Jacob,et al.  Errors and improvements in the use of archived meteorological data for chemical transport modeling , 2017 .

[11]  Yingying Jing,et al.  Global Atmospheric CO2 Concentrations Simulated by GEOS-Chem: Comparison with GOSAT, Carbon Tracker and Ground-Based Measurements , 2018 .

[12]  Bradley Zavodsky,et al.  Clouds in the Cloud: Weather Forecasts and Applications within Cloud Computing Environments , 2015 .

[13]  P. Xie,et al.  Characterization of urban CO2 column abundance with a portable low resolution spectrometer (PLRS): Comparisons with GOSAT and GEOS-Chem model data. , 2018, The Science of the total environment.

[14]  Abhishek Gupta,et al.  Evaluation of HPC Applications on Cloud , 2011, 2011 Sixth Open Cirrus Summit.

[15]  Pascal Bouvry,et al.  Amazon Elastic Compute Cloud (EC2) vs. In-House HPC Platform: A Cost Analysis , 2016, 2016 IEEE 9th International Conference on Cloud Computing (CLOUD).

[16]  Muhammad Atif,et al.  Scientific Application Performance on HPC, Private and Public Cloud Resources: A Case Study Using Climate, Cardiac Model Codes and the NPB Benchmark Suite , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum.

[17]  R. Kahn,et al.  Development and implementation of a new biomass burning emissions injection height scheme (BBEIH v1.0) for the GEOS-Chem model (v9-01-01) , 2018, Geoscientific Model Development.

[18]  G. Fox,et al.  The Future of Cloud for Academic Research Computing , 2017 .

[19]  Russell Thackston,et al.  The performance of low‐cost commercial cloud computing as an alternative in computational chemistry , 2015, J. Comput. Chem..

[20]  Armando Fox,et al.  Cloud Computing—What's in It for Me as a Scientist? , 2011, Science.

[21]  W. Brune,et al.  Global sensitivity analysis of GEOS-Chem modeled ozone and hydrogen oxides during the INTEX campaigns , 2017 .

[22]  Satoshi Matsuoka,et al.  Evaluation of HPC-Big Data Applications Using Cloud Platforms , 2017, 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[23]  Carl Boettiger,et al.  An introduction to Docker for reproducible research , 2014, OPSR.

[24]  TCO for Cloud Services: A Framework , 2015 .

[25]  David A. Patterson,et al.  Cloud Programming Simplified: A Berkeley View on Serverless Computing , 2019, ArXiv.

[26]  Dhabaleswar K. Panda,et al.  Is Singularity-based Container Technology Ready for Running MPI Applications on HPC Clouds? , 2017, UCC.

[27]  Marty Humphrey,et al.  A quantitative analysis of high performance computing with Amazon's EC2 infrastructure: The death of the local cluster? , 2009, 2009 10th IEEE/ACM International Conference on Grid Computing.

[28]  Ji-Hoon Kang,et al.  An Evaluation of the Software System Dependency of a Global Atmospheric Model , 2013 .

[29]  Edward Walker,et al.  Benchmarking Amazon EC2 for High-Performance Scientific Computing , 2008, login Usenix Mag..

[30]  Mohan K. Ramamurthy,et al.  Unlocking the Potential of NEXRAD Data through NOAA’s Big Data Partnership , 2017 .

[31]  Damien Irving,et al.  A Minimum Standard for Publishing Computational Results in the Weather and Climate Sciences , 2016 .

[32]  Meng Li,et al.  Historical (1750–2014) anthropogenic emissions of reactive gases and aerosols from the Community Emissions Data System (CEDS) , 2017 .

[33]  Jake Vanderplas,et al.  Python Data Science Handbook: Essential Tools for Working with Data , 2016 .

[34]  Carleton Coffrin,et al.  The ISTI Rapid Response on Exploring Cloud Computing 2018 , 2019, ArXiv.

[35]  Simon Ostermann,et al.  Experiences with distributed computing for meteorological applications: grid computing and cloud computing , 2015 .

[36]  Bin Wang,et al.  Bitwise identical compiling setup: prospective for reproducibility and reliability of earth system modeling , 2015 .

[37]  Nikos Parlavantzas,et al.  Efficient execution of the WRF model and other HPC applications in the cloud , 2016, Earth Science Informatics.

[38]  Vanessa Sochat,et al.  Singularity: Scientific containers for mobility of compute , 2017, PloS one.

[39]  Qunying Huang,et al.  Utilize cloud computing to support dust storm forecasting , 2013, Int. J. Digit. Earth.

[40]  Kwangwoog Jung,et al.  Performance evaluation of ROMS v3.6 on a commercial cloud system , 2017 .

[41]  Carl Boettiger,et al.  An introduction to Docker for reproducible research, with examples from the R environment , 2014, ArXiv.

[42]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[43]  Matthew Rocklin,et al.  Dask: Parallel Computation with Blocked algorithms and Task Scheduling , 2015, SciPy.

[44]  Cecelia DeLuca,et al.  The architecture of the Earth System Modeling Framework , 2003, Computing in Science & Engineering.

[45]  Laxmikant V. Kale,et al.  The who, what, why and how of high performance computing applications in the cloud , 2013 .

[46]  Carlos Maltzahn,et al.  A Containerized Mesoscale Model and Analysis Toolkit to Accelerate Classroom Learning, Collaborative Research, and Uncertainty Quantification , 2017 .

[47]  Kevin T. Pedretti,et al.  A Tale of Two Systems: Using Containers to Deploy HPC Applications on Supercomputers and Clouds , 2017, 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom).

[48]  Philippe Olivier Alexandre Navaux,et al.  High Performance Computing in the cloud: Deployment, performance and cost efficiency , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.

[49]  David M. Siuta.  Viability of Cloud-Computing for Real-Time Numerical Weather Prediction over Regional Domains , 2016 .

[50]  John Shalf,et al.  Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[51]  Constantinos Evangelinos,et al.  Cloud Computing for parallel Scientific HPC Applications: Feasibility of Running Coupled Atmosphere- , 2008 .

[52]  Justin L. Huntington,et al.  Climate Engine: Cloud Computing and Visualization of Climate and Remote Sensing Data for Advanced Natural Resource Monitoring and Process Understanding , 2017 .

[53]  Mohammad Mohammadi,et al.  Comparative benchmarking of cloud computing vendors with high performance linpack , 2017, HP3C.

[54]  Thomas L. Clune,et al.  GEOS-Chem High Performance (GCHP v11-02c): a next-generation implementation of the GEOS-Chem chemical transport model for massively parallel applications , 2018, Geoscientific Model Development.

[55]  Reid Priedhorsky,et al.  Charliecloud: Unprivileged Containers for User-Defined Software Stacks in HPC , 2017, SC17: International Conference for High Performance Computing, Networking, Storage and Analysis.

[56]  Wenguang Chen,et al.  Cloud versus in-house cluster: Evaluating Amazon cluster compute instances for running MPI applications , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[57]  Steven Pawson,et al.  HEMCO v1.0: A Versatile, ESMF-Compliant Component for Calculating Emissions in Atmospheric Models , 2014 .

[58]  Matthew J. Turk,et al.  A Scientist's Guide to Cloud Computing , 2014 .

[59]  Rupak Biswas,et al.  Performance evaluation of Amazon Elastic Compute Cloud for NASA high‐performance computing applications , 2016, Concurr. Comput. Pract. Exp..

[60]  R. Park,et al.  Efficacy of dust aerosol forecasts for East Asia using the adjoint of GEOS-Chem with ground-based observations. , 2018, Environmental pollution.

[61]  Jeffrey M. Perkel,et al.  Why Jupyter is data scientists’ computational notebook of choice , 2018, Nature.

[62]  Stephan Hoyer,et al.  xarray: N-D labeled arrays and datasets in Python , 2017 .

[63]  Tomás F. Pena,et al.  Enabling BOINC in infrastructure as a service cloud system , 2016 .

[64]  Rajkumar Buyya,et al.  HPC Cloud for Scientific and Business Applications , 2017, ACM Comput. Surv..

[65]  Bill Howe,et al.  Virtual Appliances, Cloud Computing, and Reproducible Research , 2012, Computing in Science & Engineering.

[66]  Garth N. Wells,et al.  Containers for Portable, Productive, and Performant Scientific Computing , 2016, Computing in Science & Engineering.

[67]  Lisa Gerhardt,et al.  Shifter: Containers for HPC , 2017 .

[68]  The National Earth System Prediction Capability: Coordinating the Giant , 2017 .

[69]  Martin Meyer,et al.  Imaging SKA-scale data in three different computing environments , 2015, Astron. Comput..

[70]  Zhenlong Li,et al.  Big Data and cloud computing: innovation opportunities and challenges , 2017, Int. J. Digit. Earth.

[71]  Christopher S. Lynnes,et al.  Archive Management of NASA Earth Observation Data to Support Cloud Analysis , 2017 .

[72]  Philippe Olivier Alexandre Navaux,et al.  HPC Application Performance and Cost Efficiency in the Cloud , 2017, 2017 25th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP).

[73]  May Yuan,et al.  Cloud Computing in Ocean and Atmospheric Sciences , 2016 .

[74]  Helen Shen,et al.  Interactive notebooks: Sharing the code , 2014, Nature.

[75]  Haoqiang Jin,et al.  Evaluating the Suitability of Commercial Clouds for NASA's High Performance Computing Applications: A Trade Study , 2018 .

[76]  Mehdi Raessi,et al.  The Feasibility of Amazon's Cloud Computing Platform for Parallel, GPU-Accelerated, Multiphase-Flow Simulations , 2016, Computing in Science & Engineering.