The chaos in calibrating crop models: Lessons learned from a multi-model calibration exercise

Calibration, the estimation of model parameters based on fitting the model to experimental data, is among the first steps in many applications of process-based models and has an important impact on simulated values. We propose a novel method of developing guidelines for calibration of process-based models, based on development of recommendations for calibration of the phenology component of crop models. The approach was based on a multi-model study, where all teams were provided with the same data and asked to return simulations for the same conditions. All teams were asked to document in detail their calibration approach, including choices with respect to criteria for best parameters, choice of parameters to estimate and software. Based on an analysis of the advantages and disadvantages of the various choices, we propose calibration recommendations that cover a comprehensive list of decisions and that are based on actual practices.

Qi Jing | Eric Justes | Kurt Christian Kersebaum | Daniel Wallach | Senthold Asseng | Lutz Weihermüller | Bruno Basso | Gerrit Hoogenboom | Peter J. Thorburn | Niels Schütze | Thomas Gaiser | Steven Hoek | Samuel Buis | Jørgen Eivind Olesen | Budong Qian | Zvi Hochman | Tommaso Stella | Elisabet Lewan | Camilla Dibari | Vakhtang Shelia | Per-Erik Jansson | Marco Moriondo | Marie Launay | Afshin Ghahramani | Xenia Specka | Roberto Ferrise | Thilo Streck | Thomas Wöhling | Anne Klosterhalfen | Sabine J. Seidel | Santosh Hiremath | Eckart Priesack | Hasti Nariman Zadeh | Taru Palosuo | Sebastian Gayler | Amit Kumar Srivastava | Heidi Horan | Benjamin Dumont | Johannes W. M. Pullens | Amir Souissi | Giacomo Trombi | Jing Wang | Bernardo Maestrini | Henrike Mielenz | Yan Zhu | Allard de Wit | Emmanuelle Gourdain | Mohamed Jabloun | Arne Poyda | Evelyn Wallor | Tobias K. D. Weber | Liujun Xiao | Fety Andrianasolo | Neil Crout | Cecile Garcia | Mingxia Huang | Qunying Luo | Gloria Padovan | Chuang Zhao | G. Hoogenboom | T. Palosuo | S. Buis | J. Olesen | S. Asseng | E. Justes | L. Weihermüller | Z. Hochman | H. Horan | Yan Zhu | C. Dibari | M. Moriondo | K. Kersebaum | B. Basso | P. Jansson | P. Thorburn | G. Trombi | T. Gaiser | S. Gayler | M. Jabloun | E. Priesack | T. Streck | D. Wallach | T. Stella | Qunying Luo | R. Ferrise | A. Klosterhalfen | Chuang Zhao | B. Qian | V. Shelia | Q. Jing | S. Hoek | T. Wöhling | M. Launay | A. Ghahramani | N. Crout | B. Dumont | A. D. Wit | X. Specka | E. Lewan | Liujun Xiao | J. Pullens | Jing Wang | A. Poyda | H. Mielenz | S. Seidel | B. Maestrini | N. Schütze | F. Andrianasolo | E. Wallor | E. Gourdain | Cecile Garcia | A. Souissi | S. Hiremath | G. Padovan | Mingxia Huang | A. Srivastava | Samuel Buis

[1]  Xiaolin Zhu,et al.  Plant phenology and global climate change: Current progresses and challenges , 2019, Global change biology.

[2]  Demetris Koutsoyiannis,et al.  One decade of multi-objective calibration approaches in hydrological modelling: a review , 2010 .

[3]  S. Siebert,et al.  Climate change effect on wheat phenology depends on cultivar change , 2018, Scientific Reports.

[4]  Olaf Kolditz,et al.  Calibration and Reliability in Groundwater Modelling , 2006 .

[5]  L. Samaniego,et al.  Evaluating multiple performance criteria to calibrate the distributed hydrological model of the upper Neckar catchment , 2013, Environmental Earth Sciences.

[6]  L. Ahuja,et al.  A Synthesis of Current Parameterization Approaches and Needs for Further Improvements , 2015 .

[7]  Satoshi Kawakita,et al.  Prediction and parameter uncertainty for winter wheat phenology models depend on model and parameterization method differences , 2020 .

[8]  Gerrit Hoogenboom,et al.  A sequential approach for determining the cultivar coefficients of peanut lines using end-of-season data of crop performance trials , 2008 .

[9]  Marco Franchini,et al.  Comparing several genetic algorithm schemes for the calibration of conceptual rainfall-runoff models , 1997 .

[10]  Anthony J. Jakeman,et al.  Ten iterative steps in development and evaluation of environmental models , 2006, Environ. Model. Softw..

[11]  Sabine Attinger,et al.  Multiresponse, multiobjective calibration as a diagnostic tool to compare accuracy and structural limitations of five coupled soil‐plant models and CLM3.5 , 2013 .

[12]  Henrik Madsen,et al.  Comparison of different automated strategies for calibration of rainfall-runoff models , 2002 .

[13]  Qi Jing,et al.  How well do crop modeling groups predict wheat phenology, given calibration data from the target population? , 2021 .

[14]  R. Paiva,et al.  On the contribution of remote sensing-based calibration to model hydrological and hydraulic processes in tropical regions , 2021, Journal of Hydrology.

[15]  Peter J. Thorburn,et al.  Modelling crops and cropping systems—Evolving purpose, practice and prospects , 2018, European Journal of Agronomy.

[16]  James W. Jones,et al.  GENCALC: Software to Facilitate the Use of Crop Models for Analyzing Field Experiments , 1993 .

[17]  Yvette Everingham,et al.  A theoretical and real world evaluation of two Bayesian techniques for the calibration of variety parameters in a sugarcane crop model , 2016, Environ. Model. Softw..

[18]  S. Siebert,et al.  Intensity of heat stress in winter wheat—phenology compensates for the adverse effect of global warming , 2015 .

[19]  G. Hoogenboom,et al.  Comparison of three calibration methods for modeling rice phenology , 2020 .

[20]  J. Kirkegaard,et al.  Early sowing systems can boost Australian wheat yields despite recent climate change , 2019, Nature Climate Change.

[21]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[22]  Frank Ewert,et al.  Crop modelling for integrated assessment of risk to food production from climate change , 2015, Environ. Model. Softw..

[23]  Taru Palosuo,et al.  Towards improved calibration of crop models – Where are we now and where should we go? , 2018 .

[24]  John Doherty,et al.  Efficient accommodation of local minima in watershed model calibration , 2006 .

[25]  A. Ruane,et al.  Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions , 2015, Global change biology.

[26]  G. Hoogenboom,et al.  Methodology to estimate rice genetic coefficients for the CSM-CERES-Rice model using GENCALC and GLUE genetic coefficient estimators , 2018, The Journal of Agricultural Science.

[27]  Jimmy R. Williams,et al.  Special Features of the EPIC and APEX Modeling Package and Procedures for Parameterization, Calibration, Validation, and Applications , 2015 .

[28]  Charlotte Werndl,et al.  Climate Models, Calibration, and Confirmation , 2013, The British Journal for the Philosophy of Science.

[29]  Michiel J. W. Jansen,et al.  Calibration in a Bayesian modelling framework , 2004 .

[30]  Ron Smith,et al.  Bayesian calibration of process-based forest models: bridging the gap between models and data. , 2005, Tree physiology.

[31]  Ratnesh Kumar,et al.  Coordinate descent based agricultural model calibration and optimized input management , 2020, Comput. Electron. Agric..

[32]  R. Lock,et al.  Implication of crop model calibration strategies for assessing regional impacts of climate change in Europe , 2013 .

[33]  James W. Jones,et al.  The Agricultural Model Intercomparison and Improvement Project (AgMIP): Protocols and Pilot Studies , 2013 .

[34]  C. Willmott,et al.  Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance , 2005 .

[35]  Daniel Wallach,et al.  Crop Model Calibration: A Statistical Perspective , 2011 .

[36]  Eric Justes,et al.  Methodological comparison of calibration procedures for durum wheat parameters in the STICS model , 2011 .

[37]  Fulu Tao,et al.  Simulation of maize evapotranspiration: An inter-comparison among 29 maize models , 2019, Agricultural and Forest Meteorology.

[38]  M. Donatelli,et al.  Modelling the impacts of pests and diseases on agricultural systems , 2017, Agricultural systems.

[39]  James W. Jones,et al.  Influence of likelihood function choice for estimating crop model parameters using the generalized likelihood uncertainty estimation method , 2010 .

[40]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[41]  Yu Zhang,et al.  Uncertainty in wheat phenology simulation induced by cultivar parameterization under climate warming , 2018 .

[42]  Jennifer Badham,et al.  Effective modeling for Integrated Water Resource Management: A guide to contextual practices by phases and steps and future opportunities , 2019, Environ. Model. Softw..

[43]  Comparison of Five Wheat Models Simulating Phenology under Different Sowing Dates and Varieties , 2017 .

[44]  Javed Iqbal,et al.  Understanding the DayCent model: Calibration, sensitivity, and identifiability through inverse modeling , 2015, Environ. Model. Softw..

[45]  F. Tao,et al.  Uncertainty from model structure is larger than that from model parameters in simulating rice phenology in China , 2017 .

[46]  Qi Jing,et al.  Multi-model evaluation of phenology prediction for wheat in Australia , 2021 .

[47]  Keith Beven,et al.  Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology , 2001 .

[48]  David R. Anderson,et al.  AIC model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons , 2011, Behavioral Ecology and Sociobiology.

[49]  J. Zadoks.  A decimal code for the growth stages of cereals , 1974 .

[50]  James W. Jones,et al.  A dynamic model with QTL covariables for predicting flowering time of common bean (Phaseolus vulgaris) genotypes , 2018, European Journal of Agronomy.

[51]  John Doherty,et al.  Two statistics for evaluating parameter identifiability and error reduction , 2009 .

[52]  James O. Berger,et al.  The interplay of Bayesian and frequentist analysis , 2004 .