Sensitivity analysis of a sensitivity analysis: We are likely overlooking the impact of distributional assumptions

Although uncertainty in input factor distributions is known to affect sensitivity analysis (SA) results, a standard procedure to quantify its impact is not available. We addressed this problem by performing a SA (generating sample of parameter distributions) of a SA (generating samples of parameter values for each generated distribution) of the WARM rice model using the Sobol’ method. The sample of distributions was generated using distributions of jackknife statistics calculated on literature values. This allowed mimicking the differences in distributions that could derive from different selection of literature sources. Despite the very low plasticity of WARM, the ranks of the two most relevant parameters was overturned in 22% of the cases and, in general, differed from what achieved in earlier SAs performed on the same model under similar conditions. SA results were mainly affected by uncertainty in distribution of parameters involved in non-linear effects or interacting with others. The procedure identified parameters whose uncertainty in distribution can alter SA results, i.e., parameters whose distributions could need to be refined.

[1]  M. H. Quenouille Approximate Tests of Correlation in Time‐Series , 1949 .

[2]  Stefano Tarantola,et al.  Sensitivity analysis in model calibration: GSA-GLUE approach , 2001 .

[3]  K. Chenu,et al.  Assessment of the Potential Impacts of Wheat Plant Traits across Environments by Combining Crop Modeling and Global Sensitivity Analysis , 2015, PloS one.

[4]  Lin Qiu,et al.  Evaluation of WARM for different establishment techniques in Jiangsu (China) , 2014 .

[5]  H. H. Laar,et al.  ORYZA 1 : an ecophysiological model for irrigated rice production , 1994 .

[6]  Stefano Tarantola,et al.  Sensitivity analysis of the rice model WARM in Europe: Exploring the effects of different locations, climates and methods of analysis on model sensitivity to crop parameters , 2010, Environ. Model. Softw..

[7]  M. Semenov,et al.  In silico system analysis of physiological traits determining grain yield and protein concentration for wheat as influenced by climate and crop management , 2015, Journal of experimental botany.

[8]  Jing Wang,et al.  Parameter sensitivity analysis of crop growth models based on the extended Fourier Amplitude Sensitivity Test method , 2013, Environ. Model. Softw..

[9]  J. Heilman,et al.  Seasonal variation in radiation use efficiency of irrigated rice , 2001 .

[10]  T. Horie,et al.  Studies on Crop-Weather Relationship Model in Rice , 1985 .

[11]  M. Dingkuhn,et al.  Reserve mobilization, dry matter partitioning and specific leaf area in seedlings of African rice cultivars differing in early vigor , 1999 .

[12]  Claudio O. Stöckle,et al.  CropSyst, a cropping systems simulation model: Water/nitrogen budgets and crop yield☆ , 1994 .

[13]  J. Monteith Light Interception and Radiative Exchange in Crop Stands , 1969 .

[14]  S. Uhlenbrook,et al.  Sensitivity analyses of a distributed catchment model to verify the model structure , 2005 .

[15]  K. Terashima,et al.  A Varietal Difference in Coleoptile Growth is Correlated with Seedling Establishment of Direct Seeded Rice in Submerged Field under Low-Temperature Conditions , 2001 .

[16]  Xinyou Yin,et al.  The Effect of Temperature on Leaf Appearance in Rice , 1996 .

[17]  Senthold Asseng,et al.  Simulation of grain protein content with APSIM-Nwheat , 2002 .

[18]  Francesca Pianosi,et al.  Global Sensitivity Analysis of environmental models: Convergence and validation , 2016, Environ. Model. Softw..

[19]  A. Saltelli,et al.  Sensitivity analysis for chemical models. , 2005, Chemical reviews.

[20]  Gianni Bellocchi,et al.  Multi-Metric Evaluation of the Models WARM, CropSyst, and WOFOST for Rice , 2009 .

[21]  Toshichika Iizumi,et al.  Probabilistic evaluation of climate change impacts on paddy rice productivity in Japan , 2011 .

[22]  Gianni Bellocchi,et al.  Comparison of sensitivity analysis techniques: A case study with the rice model WARM , 2010 .

[23]  Jeffrey G. Arnold,et al.  Rice Parameters Describing Crop Performance of Four U.S. Cultivars , 2001 .

[24]  G. Richter,et al.  Development of a sink–source interaction model for the growth of short-rotation coppice willow and in silico exploration of genotype×environment effects , 2015, Journal of experimental botany.

[25]  M. Ohnishi,et al.  Physiological characteristics of high-yielding rice inferred from cross-location experiments , 1997 .

[26]  Andrej Ceglar,et al.  The simulation of phenological development in dynamic crop model: The Bayesian comparison of different methods , 2011 .

[27]  Michaël Dingkuhn,et al.  Relationships between upland rice canopy characteristics and weed competitiveness , 1999 .

[28]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[29]  J. Goudriaan,et al.  Monitoring rice reflectance at field level for estimating biomass and LAI , 1998 .

[30]  P. Reed,et al.  Sensitivity-guided reduction of parametric dimensionality for multi-objective calibration of watershed models , 2009 .

[31]  D. Charles-Edwards,et al.  Physiological Determinants of Crop Growth. , 1986 .

[32]  C. Fortuin,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory , 1973 .

[33]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[34]  Jim W. Hall,et al.  Sensitivity analysis of environmental models: A systematic review with practical workflow , 2014, Environ. Model. Softw..

[35]  Stefano Tarantola,et al.  SAMO 2001: methodological advances and innovative applications of sensitivity analysis , 2003, Reliability Engineering & System Safety.

[36]  Stefano Bocchi,et al.  Evaluation of CropSyst for simulating the yield of flooded rice in northern Italy , 2005 .

[37]  H. van Keulen,et al.  Modelling of agricultural production: Weather, soils, and crops , 1986 .

[38]  S. Koutroubas,et al.  Grain quality variation and relationships with morpho-physiological traits in rice (Oryza sativa L.) genetic resources in Europe , 2004 .

[39]  W. Cheng,et al.  Differential response of rice plants to high night temperatures imposed at varying developmental phases , 2015 .

[40]  Xin-Guang Zhu,et al.  Improving photosynthetic efficiency for greater yield. , 2010, Annual review of plant biology.

[41]  A. Saltelli,et al.  A quantitative model-independent method for global sensitivity analysis of model output , 1999 .

[42]  Anthony J. Jakeman,et al.  Ten iterative steps in development and evaluation of environmental models , 2006, Environ. Model. Softw..

[43]  S. Stearns,et al.  The Evolution of Life Histories , 1992 .

[44]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[45]  H. Nakagawa,et al.  Effects of Elevated CO2 Concentration and High Temperature on Growth and Yield of Rice : II. The effect on yield and its components of Akihikari rice , 1996 .

[46]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[47]  Marco Acutis,et al.  Quantifying plasticity in simulation models , 2012 .

[48]  Willy Bauwens,et al.  Sobol' sensitivity analysis of a complex environmental model , 2011, Environ. Model. Softw..

[49]  Kazuki Saito,et al.  Plant Characteristics of High-Yielding Upland Rice Cultivars in West Africa , 2016 .

[50]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[51]  R. Iman,et al.  A measure of top-down correlation , 1987 .

[52]  Masahiro Kasuya,et al.  Examination of the water balance of irrigated paddy fields in SWAT 2009 using the curve number procedure and the pothole module , 2014 .

[53]  B. Croke,et al.  Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R , 2013 .