Multi-Metric Evaluation of the Models WARM, CropSyst, and WOFOST for Rice

WARM (Water Accounting Rice Model) simulates paddy rice (Oryza sativa L.), based on temperature-driven development and radiation-driven crop growth. It also simulates: biomass partitioning, floodwater effect on temperature, spikelet sterility, floodwater and chemicals management, and soil hydrology. Biomass estimates from WARM were evaluated and compared with the ones from two generic crop models (CropSyst, WOFOST). The test-area was the Po Valley (Italy). Data collected at six sites from 1989 to 2004 from rice crops grown under flooded and non-limiting conditions were split into a calibration (to estimate some model parameters) and a validation set. For model evaluation, a fuzzy-logic based multiple-metrics indicator (MQI) was used: 0 (best) ≤ MQI ≤ 1 (worst). WARM estimates compared well with the actual data (mean MQI = 0.037 against 0.167 and 0.173 with CropSyst and WOFOST, respectively). On an average, the three models performed similarly for individual validation metrics such as modelling efficiency (EF > 0.90) and correlation coefficient (R > 0.98). WARM performed best in a weighed measure of the Akaike Information Criterion: (worst) 0<wk<10<wk<1 (best), considering estimation accuracy and number of parameters required to achieve it (mean wk=0.983wk=0.983 against 0.007 and ∼0.000 with CropSyst and WOFOST, respectively). WARM results were sensitive to 30% of the model parameters (ratio being lower with both CropSyst, <10%, and WOFOST, <20%), but appeared the easiest model to use because of the lowest number of crop parameters required (10 against 15 and 34 with CropSyst and WOFOST, respectively). This study provides a concrete example of the possibilities offered using a range of assessment metrics to evaluate model estimates, predictive capabilities, and complexity.

[1]  Daniela Stroppiana,et al.  Analysis of rice sample size variability due to development stage, nitrogen fertilization, sowing technique and variety using the visual jackknife , 2006 .

[2]  Luigi Mariani,et al.  Analysis and modelling of water and near water temperatures in flooded rice (Oryza sativa L.) , 2005 .

[3]  H. van Keulen,et al.  The 'School of de Wit' crop growth simulation models: a pedigree and historical overview. , 1996 .

[4]  J. Marques,et al.  CRISP (crayfish and rice integrated system of production): 1. Modelling rice (Oryza sativa) growth and production , 1999 .

[5]  Gianni Bellocchi,et al.  An indicator of solar radiation model performance based on a fuzzy expert system , 2002 .

[6]  Gianni Bellocchi,et al.  An indicator to evaluate the environmental Impact of olive oil waste water’s shedding on cultivated fields , 2006 .

[7]  Paul Teng,et al.  Systems approaches for agricultural development , 1993, Systems Approaches for Sustainable Agricultural Development.

[8]  Luca Bechini,et al.  Parameterization of a crop growth and development simulation model at sub-model components level. An example for winter wheat (Triticum aestivum L.) , 2006, Environ. Model. Softw..

[9]  Bellocchi Gianni,et al.  An indicator to Evaluate the Environmental Impact of Olive Oil Waste Water\'s Shedding on Cultivated Fields , 2006 .

[10]  Luca Bechini,et al.  A preliminary evaluation of the simulation model CropSyst for alfalfa , 2004 .

[11]  Howard M. Taylor,et al.  Water Use in Agriculture. (Book Reviews: Limitations to Efficient Water Use in Crop Production) , 1984 .

[12]  M. Salam,et al.  Comparing Simulated and Measured Values Using Mean Squared Deviation and its Components , 2000 .

[13]  R. Blanchet,et al.  Radiation-use efficiency in biomass accumulation prior to grain-filling for five grain-crop species , 1989 .

[14]  H. H. Laar,et al.  ORYZA 1 : an ecophysiological model for irrigated rice production , 1994 .

[15]  Xinyou Yin,et al.  The Effect of Temperature on Leaf Appearance in Rice , 1996 .

[16]  M. Dingkuhn,et al.  Climatic determinants of irrigated rice performance in the Sahel — I. Photothermal and micro-climatic responses of flowering , 1995 .

[17]  J. Heilman,et al.  Seasonal variation in radiation use efficiency of irrigated rice , 2001 .

[18]  Stefano Bocchi,et al.  The CropSyst Model to Simulate the N Balance of Rice for Alternative Management , 2006 .

[19]  C. Willmott.  Some Comments on the Evaluation of Model Performance , 1982 .

[20]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[21]  J. Monteith.  Climate and the efficiency of crop production in Britain , 1977 .

[22]  Claudio O. Stöckle,et al.  Simulation of water uptake in maize, using different levels of process detail , 1999 .

[23]  Rezaul Mahmood,et al.  Air temperature variations and rice productivity in Bangladesh: a comparative study of the performance of the YIELD and the CERES-Rice models , 1998 .

[24]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice , 2002 .

[25]  D. Bachelet.  The impacts of climate change on rice yield: a comparison of four model performances , 1993 .

[26]  M. Rivington,et al.  Evaluation of three model estimations of solar radiation at 24 UK stations , 2005 .

[27]  J. Porter,et al.  A comparison of the models AFRCWHEAT2, CERES-Wheat, Sirius, SUCROS2 and SWHEAT with measurements from wheat grown under drought , 1998 .

[28]  David Favis-Mortlock,et al.  The Limits of Erosion Modeling , 2001 .

[29]  T. Warren Liao,et al.  A fuzzy c‐medians variant for the generation of fuzzy term sets , 2002, Int. J. Intell. Syst..

[30]  W Day.  Modelling crop physiology for integrated decision making , 2001 .

[31]  Michaël Dingkuhn,et al.  Relationships between upland rice canopy characteristics and weed competitiveness , 1999 .

[32]  P. Aggarwal.  Uncertainties in crop, soil and weather inputs used in growth models: Implications for simulated outputs and their applications , 1995 .

[33]  A. R. Saka,et al.  Maize modeling in Malawi: a tool for soil fertility research and development , 1993 .

[34]  C. Stöckle,et al.  CropSyst, a cropping systems simulation model , 2003 .

[35]  J. Goudriaan,et al.  Monitoring rice reflectance at field level for estimating biomass and LAI , 1998 .

[36]  Claudio O. Stöckle,et al.  Evaluation of CropSyst for cropping systems at two locations of northern and southern Italy , 1997 .

[37]  Marc J. Mazerolle.  Mouvements et reproduction des amphibiens en tourbières perturbées , 2004 .

[38]  Michael J. Savage,et al.  Comparison of estimates of daily solar radiation from air temperature range for application in crop simulations , 2008 .

[39]  Gilles Eggenspieler,et al.  Numerical Simulation of Pollutant Emissionand Flame Extinction in Lean Premixed Systems , 2005 .

[40]  L. S. Pereira,et al.  Crop evapotranspiration : guidelines for computing crop water requirements , 1998 .

[41]  Stefano Bocchi,et al.  Evaluation of CropSyst for simulating the yield of flooded rice in northern Italy , 2005 .

[42]  A. Jakeman,et al.  How much complexity is warranted in a rainfall‐runoff model? , 1993 .

[43]  H. van Keulen,et al.  Modelling of agricultural production: Weather, soils, and crops , 1986 .

[44]  Using CropSyst to Simulate Spring Wheat Growth in Black Soil Zone of Northeast China , 2006 .

[45]  K. Ishihara,et al.  Science of the rice plant , 1993 .

[46]  R. C. Muchow,et al.  Radiation Use Efficiency , 1999 .

[47]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[48]  H. Akaike.  A new look at the statistical model identification , 1974 .

[49]  Confalonieri Roberto,et al.  Exploratory Sensitivity Analysis of CropSyst, WARM and WOFOST: a Case-Study with Rice Biomass Simulations , 2006 .

[50]  M. Dingkuhn,et al.  Growth and yield potential of Oryza sativa and O. glaberrima upland rice cultivars and their interspecific progenies , 1998 .

[51]  Gianni Bellocchi,et al.  IRENE_DLL: A Class Library for Evaluating Numerical Estimates , 2003 .

[52]  Michio Sugeno,et al.  An introductory survey of fuzzy control , 1985, Inf. Sci..

[53]  Gianni Bellocchi,et al.  Modelling solar radiation over complex terrains using monthly climatological data , 2007 .

[54]  Luis Orlindo Tedeschi,et al.  Assessment of the adequacy of mathematical models , 2006 .

[55]  V. Sadras,et al.  Fallow soil evaporation and water storage as affected by stubble in sub-humid (Argentina) and semi-arid (Australia) environments , 2006 .

[56]  Marc J. Mazerolle,et al.  APPENDIX 1: Making sense out of Akaike's Information Criterion (AIC): its use and interpretation in model selection and inference from ecological data , 2007 .

[57]  K. Loague,et al.  Statistical and graphical methods for evaluating solute transport models: Overview and application , 1991 .

[58]  H. van Keulen,et al.  A summary model for crop growth , 1982 .

[59]  Senthold Asseng,et al.  An overview of APSIM, a model designed for farming systems simulation , 2003 .