The International Land Model Benchmarking (ILAMB) System: Design, Theory, and Implementation

Author(s): Collier, N; Hoffman, FM; Lawrence, DM; Keppel-Aleks, G; Koven, CD; Riley, WJ; Mu, M; Randerson, JT | Abstract: ©2018. The Authors. The increasing complexity of Earth system models has inspired efforts to quantitatively assess model fidelity through rigorous comparison with best available measurements and observational data products. Earth system models exhibit a high degree of spread in predictions of land biogeochemistry, biogeophysics, and hydrology, which are sensitive to forcing from other model components. Based on insights from prior land model evaluation studies and community workshops, the authors developed an open source model benchmarking software package that generates graphical diagnostics and scores model performance in support of the International Land Model Benchmarking (ILAMB) project. Employing a suite of in situ, remote sensing, and reanalysis data sets, the ILAMB package performs comprehensive model assessment across a wide range of land variables and generates a hierarchical set of web pages containing statistical analyses and figures designed to provide the user insights into strengths and weaknesses of multiple models or model versions. Described here is the benchmarking philosophy and mathematical methodology embodied in the most recent implementation of the ILAMB package. Comparison methods unique to a few specific data sets are presented, and guidelines for configuring an ILAMB analysis and interpreting resulting model performance scores are discussed. ILAMB is being adopted by modeling teams and centers during model development and for model intercomparison projects, and community engagement is sought for extending evaluation metrics and adding new observational data sets to the benchmarking framework.

[1]  Michael G. Bosilovich,et al.  Comparing evaporative sources of terrestrial precipitation and their extremes in MERRA using relative entropy , 2014 .

[2]  James A. Westfall,et al.  NACP Aboveground Biomass and Carbon Baseline Data, V.2 (NBCD 2000), U.S.A., 2000 , 2013 .

[3]  R. Schnur,et al.  Climate-carbon cycle feedback analysis: Results from the C , 2006 .

[4]  Charles Doutriaux,et al.  A More Powerful Reality Test for Climate Models , 2016 .

[5]  Gab Abramowitz,et al.  Towards a public, standardized, diagnostic benchmarking system for land surface models , 2012 .

[6]  S. Seneviratne,et al.  Recent decline in the global land evapotranspiration trend due to limited moisture supply , 2010, Nature.

[7]  Ranga B. Myneni,et al.  Estimation of global leaf area index and absorbed par using radiative transfer models , 1997, IEEE Trans. Geosci. Remote. Sens..

[8]  N. Loeb,et al.  Surface Irradiances Consistent With CERES-Derived Top-of-Atmosphere Shortwave and Longwave Irradiances , 2013 .

[9]  P. Xie,et al.  Global Precipitation: A 17-Year Monthly Analysis Based on Gauge Observations, Satellite Estimates, and Numerical Model Outputs , 1997 .

[10]  Marc Simard,et al.  A comprehensive benchmarking system for evaluating global vegetation models , 2012 .

[11]  J. Gregory,et al.  Quantifying Carbon Cycle Feedbacks , 2009 .

[12]  Kenton McHenry,et al.  A quantitative assessment of a terrestrial biosphere model's data needs across North American biomes , 2014 .

[13]  R. Dickinson,et al.  Couplings between changes in the climate system and biogeochemistry , 2007 .

[14]  W. Salas,et al.  Benchmark map of forest carbon stocks in tropical regions across three continents , 2011, Proceedings of the National Academy of Sciences.

[15]  Karl E. Taylor,et al.  An overview of CMIP5 and the experiment design , 2012 .

[16]  John F. B. Mitchell,et al.  THE WCRP CMIP3 Multimodel Dataset: A New Era in Climate Change Research , 2007 .

[17]  Michael C. Dietze,et al.  Facilitating feedbacks between field measurements and ecosystem models , 2013 .

[18]  T. Holmes,et al.  Global land-surface evaporation estimated from satellite-based observations , 2010 .

[19]  Charles Doutriaux,et al.  Performance metrics for climate models , 2008 .

[20]  J. Randerson,et al.  Multicentury changes in ocean and land contributions to the climate‐carbon feedback , 2015 .

[21]  P. Jones,et al.  Updated high‐resolution grids of monthly climatic observations – the CRU TS3.10 Dataset , 2014 .

[22]  K.,et al.  Carbon–Concentration and Carbon–Climate Feedbacks in CMIP5 Earth System Models , 2012 .

[23]  Kenneth W. Harrison,et al.  Land surface Verification Toolkit (LVT) – a generalized framework for land surface model evaluation , 2012 .

[24]  S. Swenson,et al.  Post‐processing removal of correlated errors in GRACE data , 2006 .

[25]  J. Randerson,et al.  Causes and implications of persistent atmospheric carbon dioxide biases in Earth System Models , 2013 .

[26]  Kevin E. Trenberth,et al.  Estimates of Freshwater Discharge from Continents: Latitudinal and Seasonal Variations , 2002 .

[27]  S. Los,et al.  A comprehensive set of benchmark tests for a land surface model of simultaneous fluxes of water and carbon at both the global and seasonal scale , 2010 .

[28]  Kenneth L. Denman Canada.  Couplings between changes in the climate system and biogeochemistry , 2008 .

[29]  John F. B. Mitchell,et al.  THE WCRP CMIP 3 MULTIMODEL DATASET A New Era in Climate Change Research , 2017 .

[30]  Pierre Friedlingstein,et al.  Uncertainties in CMIP5 Climate Projections due to Carbon Cycle Feedbacks , 2014 .

[31]  J. Thepaut,et al.  The ERA‐Interim reanalysis: configuration and performance of the data assimilation system , 2011 .

[32]  P. Cox,et al.  Evaluating the Land and Ocean Components of the Global Carbon Cycle in the CMIP5 Earth System Models , 2013 .

[33]  Daniel M. Ricciuto,et al.  Predicting long‐term carbon sequestration in response to CO2 enrichment: How and why do current ecosystem models differ? , 2015 .

[34]  J. Randerson,et al.  Representing leaf and root physiological traits in CLM improves global carbon and nitrogen cycling predictions , 2016 .

[35]  Philippe Ciais,et al.  A framework for benchmarking land models , 2012 .

[36]  R. Betts,et al.  Acceleration of global warming due to carbon-cycle feedbacks in a coupled climate model , 2000, Nature.

[37]  P. Ciais,et al.  Benchmarking coupled climate‐carbon models against long‐term atmospheric CO2 measurements , 2010 .

[38]  Atul K. Jain,et al.  Global Carbon Budget 2016 , 2016 .

[39]  A. Stuart,et al.  Data Assimilation: A Mathematical Introduction , 2015, 1506.07825.

[40]  Scott C. Doney,et al.  Sustained climate warming drives declining marine biological productivity , 2018, Science.

[41]  K. Lindsay,et al.  Evolution of carbon sinks in a changing climate. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Simon Read,et al.  ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP , 2015 .

[43]  Philip Lewis,et al.  An assessment of the MODIS collection 5 leaf area index product for a region of mixed coniferous forest , 2011 .

[44]  Gabriel Abramowitz,et al.  Towards a benchmark for land surface models , 2005 .

[45]  J. Randerson,et al.  Assessing variability and long-term trends in burned area by merging multiple satellite fire products , 2009 .

[46]  Zong-Liang Yang,et al.  Technical description of version 4.5 of the Community Land Model (CLM) , 2013 .

[47]  K. Taylor.  Summarizing multiple aspects of model performance in a single diagram , 2001 .

[48]  T. Reichler,et al.  How Well Do Coupled Models Simulate Today's Climate? , 2008 .

[49]  Robert F. Adler,et al.  Estimating Climatological Bias Errors for the Global Precipitation Climatology Project (GPCP) , 2012 .

[50]  C. Koven,et al.  Multiple soil nutrient competition between plants, microbes, and mineral surfaces: model development, parameterization, and example applications in several tropical forests , 2015 .

[51]  A. Arneth,et al.  Separation of net ecosystem exchange into assimilation and respiration using a light response curve approach: critical issues and global evaluation , 2010 .

[52]  M. Lomas,et al.  Evaluation of terrestrial carbon cycle models for their response to climate variability and to CO2 trends , 2013, Global change biology.

[53]  J. Randerson,et al.  Systematic assessment of terrestrial biogeochemistry in coupled climate–carbon models , 2009 .

[54]  Sönke Zaehle,et al.  Towards a more objective evaluation of modelled land-carbon trends using atmospheric CO 2 and satellite-based vegetation activity observations , 2013 .

[55]  B. Elberling,et al.  A new data set for estimating organic carbon storage to 3 m depth in soils of the northern circumpolar permafrost region , 2013 .

[56]  W. Riley,et al.  Meta-analysis of high-latitude nitrogen-addition and warming studies implies ecological mechanisms overlooked by land models , 2014 .

[57]  Gert König-Langlo,et al.  The Baseline Surface Radiation Network and its World Radiation Monitoring Centre at the Alfred Wegener Institute , 2013 .

[58]  A. Ward,et al.  GRACE: Gravity Recovery and Climate Experiment , 2002 .

[59]  Atul K. Jain,et al.  Comprehensive ecosystem model‐data synthesis using multiple data sets at two temperate forest free‐air CO2 enrichment experiments: Model performance at ambient CO2 concentration , 2014 .

[60]  P. Dirmeyer,et al.  The Plumbing of Land Surface Models: Benchmarking Model Performance , 2015 .

[61]  Atul K. Jain,et al.  Evaluation of 11 terrestrial carbon–nitrogen cycle models against observations from two temperate Free-Air CO2 Enrichment studies , 2014, The New phytologist.

[62]  U. Schneider,et al.  GPCC's new land surface precipitation climatology based on quality-controlled in situ data and its role in quantifying the global water cycle , 2013, Theoretical and Applied Climatology.

[63]  J. Randerson,et al.  Causes of variation in soil carbon simulations from CMIP5 Earth system models and comparison with observations , 2012 .

[64]  J. Dufresne,et al.  Positive feedback between future climate change and the carbon cycle , 2001 .

[65]  J. Randerson,et al.  Interactions between land use change and carbon cycle feedbacks , 2017 .

[66]  M. D. Nelson,et al.  Mapping U.S. forest biomass using nationwide forest inventory data and moderate resolution information , 2008 .

[67]  Nathan Collier,et al.  Understanding the representativeness of FLUXNET for upscaling carbon flux from eddy covariance measurements , 2016 .