Assessing artificial neural networks and statistical methods for infilling missing soil moisture records

Summary Soil moisture information is critically important for water management operations including flood forecasting, drought monitoring, and groundwater recharge estimation. While an accurate and continuous record of soil moisture is required for these applications, the available soil moisture data, in practice, is typically fraught with missing values. There are a wide range of methods available to infilling hydrologic variables, but a thorough inter-comparison between statistical methods and artificial neural networks has not been made. This study examines 5 statistical methods including monthly averages, weighted Pearson correlation coefficient, a method based on temporal stability of soil moisture, and a weighted merging of the three methods, together with a method based on the concept of rough sets. Additionally, 9 artificial neural networks are examined, broadly categorized into feedforward, dynamic, and radial basis networks. These 14 infilling methods were used to estimate missing soil moisture records and subsequently validated against known values for 13 soil moisture monitoring stations for three different soil layer depths in the Yanco region in southeast Australia. The evaluation results show that the top three highest performing methods are the nonlinear autoregressive neural network, rough sets method, and monthly replacement. A high estimation accuracy (root mean square error (RMSE) of about 0.03 m 3 / m 3 ) was found in the nonlinear autoregressive network, due to its regression based dynamic network which allows feedback connections through discrete-time estimation. An equally high accuracy (0.05 m 3 / m 3 RMSE) in the rough sets procedure illustrates the important role of temporal persistence of soil moisture, with the capability to account for different soil moisture conditions.

[1]  R. Young,et al.  Calibration of a Soil Moisture Sensor in Heterogeneous Terrain , 2007 .

[2]  Ashish Sharma,et al.  A study of optimal model lag and spatial inputs to artificial neural network for rainfall forecasting , 2000 .

[3]  A. Berg,et al.  Streamflow predictability in the Saskatchewan/Nelson River basin given macroscale estimates of the initial soil moisture status , 2006 .

[4]  Z. Pawlak Rough set approach to knowledge-based decision support , 1997 .

[5]  Omaima N. A. AL-Allaf,et al.  NONLINEAR AUTOREGRESSIVE NEURAL NETWORK FOR ESTIMATION SOIL TEMPERATURE: A COMPARISON OF DIFFERENT OPTIMIZATION NEURAL NETWORK ALGORITHMS , 2011 .

[6]  José Martínez-Fernández,et al.  Temporal Stability of Soil Moisture in a Large‐Field Experiment in Spain , 2003 .

[7]  J. Martínez-Fernández,et al.  Mean soil moisture estimation using temporal stability analysis , 2005 .

[8]  N. Null Artificial Neural Networks in Hydrology. I: Preliminary Concepts , 2000 .

[9]  Luca Brocca,et al.  Soil moisture temporal stability over experimental areas in Central Italy. , 2009 .

[10]  Arash Andalib,et al.  Forecasting the Unknown Dynamics in NN3 Database Using a Nonlinear Autoregressive Recurrent Neural Network , 2007, 2007 International Joint Conference on Neural Networks.

[11]  Andrew W. Western,et al.  Towards a general equation for frequency domain reflectometers | NOVA. The University of Newcastle's Digital Repository , 2010 .

[12]  Gift Dumedah,et al.  Evaluation of statistical methods for infilling missing values in high-resolution soil moisture data , 2011 .

[13]  Luca Brocca,et al.  Spatial‐temporal variability of soil moisture and its estimation across scales , 2010 .

[14]  Jerzy W. Grzymala-Busse,et al.  Rough sets : New horizons in commercial and industrial AI , 1995 .

[15]  Lei Tian,et al.  Estimation of spatial soil moisture averages in a large gully of the Loess Plateau of China through statistical and modeling solutions , 2013 .

[16]  Paulin Coulibaly,et al.  Comparison of neural network methods for infilling missing daily weather records , 2007 .

[17]  Aleksander Øhrn,et al.  Discernibility and Rough Sets in Medicine: Tools and Applications , 2000 .

[18]  W. J. Shuttleworth,et al.  Integration of soil moisture remote sensing and hydrologic modeling using data assimilation , 1998 .

[19]  W. C. Lennox,et al.  Comparative Studies in Problems of Missing Extreme Daily Streamflow Records , 2009 .

[20]  null null,et al.  Artificial Neural Networks in Hydrology. II: Hydrologic Applications , 2000 .

[21]  Witold F. Krajewski,et al.  Rainfall forecasting in space and time using a neural network , 1992 .

[22]  Martin T. Hagan,et al.  Neural network design , 1995 .

[23]  G. Vachaud,et al.  Temporal Stability of Spatially Measured Soil Water Probability Density Function , 1985 .

[24]  Gift Dumedah,et al.  Minimizing the effects of inaccurate sediment description in borehole data using rough sets and transition probability , 2008, J. Geogr. Syst..

[25]  Omaima N. A. AL-Allaf Cascade-Forward vs. Function Fitting Neural Network for Improving Image Quality and Learning Time in Image Compression System , 2012 .

[26]  G. Heathman,et al.  Temporal stability of surface soil moisture in the Little Washita River watershed and its applications in satellite soil moisture product validation , 2006 .

[27]  Richard Parry,et al.  The task ahead , 1948 .

[28]  John O. Odiyo,et al.  Filling of missing rainfall data in Luvuvhu River Catchment using artificial neural networks , 2011 .

[29]  R. Mahmood Scale issues in soil moisture modelling: problems and prospects , 1996 .

[30]  Gift Dumedah,et al.  Minimizing effects of scale distortion for spatially grouped census data using rough sets , 2008, J. Geogr. Syst..

[31]  T. Chow,et al.  Nonlinear autoregressive integrated neural network model for short-term load forecasting , 1996 .

[32]  Calibration of the Murrumbidgee Monitoring Network CS616 Soil Moisture Sensors , 2008 .

[33]  Thomas M. Smith,et al.  An Improved In Situ and Satellite SST Analysis for Climate , 2002 .

[34]  Yi Liu,et al.  A three-dimensional gap filling method for large geophysical datasets: Application to global satellite soil moisture observations , 2012, Environ. Model. Softw..

[35]  Dimitri P. Solomatine,et al.  Application of adaptive fuzzy rule-based models for reconstruction of missing precipitation events , 2000 .

[36]  Adebayo Adeloye,et al.  Infilling of missing rainfall and streamflow data in the Shire River basin, Malawi – A self organizing map approach , 2012 .

[37]  T. Jackson,et al.  Temporal persistence and stability of surface soil moisture in a semi-arid watershed , 2008 .

[38]  R Govindaraju,et al.  ARTIFICIAL NEURAL NETWORKS IN HYDROLOGY: II, HYDROLOGIC APPLICATIONS , 2000 .

[39]  Sorab Panday,et al.  Future of Groundwater Modeling , 2012, Ground water.

[40]  J. Srinivas,et al.  Neural Networks: Algorithms and Applications , 2002 .

[41]  T. Schneider Analysis of Incomplete Climate Data: Estimation of Mean Values and Covariance Matrices and Imputation of Missing Values. , 2001 .

[42]  Delphis F. Levia,et al.  Soil moisture: A central and unifying theme in physical geography , 2011 .

[43]  Slobodan P. Simonovic,et al.  Group-based estimation of missing hydrological data: II. Application to streamflows , 2000 .

[44]  Kevin E. Trenberth,et al.  Evaluation of the atmospheric moisture and hydrological cycle in the NCEP/NCAR reanalyses , 1998 .

[45]  Andrew W. Western,et al.  A calibration and temperature correction procedure for the water‐content reflectometer , 2005 .

[46]  J. P. King,et al.  Infilling Missing Daily Evapotranspiration Data Using Neural Networks , 2010 .

[47]  C. W. Thornthwaite THE TASK AHEAD1 , 1961 .

[48]  Mark Beale,et al.  Neural Network Toolbox™ User's Guide , 2015 .

[49]  A. B. Smith,et al.  The Murrumbidgee soil moisture monitoring network data set , 2012 .

[50]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[51]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[52]  W. W. Ng,et al.  Comparisons of traditional and novel stochastic models for the generation of daily precipitation occurrences , 2010 .

[53]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..