Comment on ``Two statistics for evaluating parameter identifiability and error reduction'' by John Doherty and Randall J. Hunt

Doherty and Hunt (2009) present important ideas for first-order-second moment sensitivity analysis, but five issues are discussed in this comment. First, considering the composite-scaled sensitivity (CSS) jointly with parameter correlation coefficients (PCC) in a CSS/PCC analysis addresses the difficulties with CSS mentioned in the introduction. Second, their new parameter identifiability statistic actually is likely to do a poor job of parameter identifiability in common situations. The statistic instead performs the very useful role of showing how model parameters are included in the estimated singular value decomposition (SVD) parameters. Its close relation to CSS is shown. Third, the idea from p. 125 that a suitable truncation point for SVD parameters can be identified using the prediction variance is challenged using results from Moore and Doherty (2005). Fourth, the relative error reduction statistic of Doherty and Hunt is shown to belong to an emerging set of statistics here named perturbed calculated variance statistics. Finally, the perturbed calculated variance statistics OPR and PPR mentioned on p. 121 are shown to explicitly include the parameter null-space component of uncertainty. Indeed, OPR and PPR results that account for null-space uncertainty have appeared in the literature since 2000.

[1]  R. Hunt,et al.  Are Models Too Simple? Arguments for Increased Parameterization , 2007, Ground water.

[2]  Mary C Hill,et al.  Determining Extreme Parameter Correlation in Ground Water Models , 2003, Ground water.

[3]  H. C. Barlebo,et al.  Identification of groundwater parameters at Columbus, Mississippi, using a 3D inverse flow and transport model , 1996 .

[4]  Mary C Hill,et al.  Numerical methods for improving sensitivity analysis and parameter estimation of virus transport simulated using sorptive-reactive processes. , 2005, Journal of contaminant hydrology.

[5]  Mary C. Hill,et al.  Data error and highly parameterized groundwater models , 2008 .

[6]  Mary C. Hill,et al.  Sensitivity analysis, calibration, and testing of a distributed hydrological model using error‐based weighting and one objective function , 2009 .

[7]  K. Bencala,et al.  Automated calibration of a stream solute transport model: implications for interpretation of biogeochemical parameters , 2003, Journal of the North American Benthological Society.

[8]  S. P. Neuman,et al.  On model selection criteria in multimodel analysis , 2007 .

[9]  C. Welty,et al.  Stochastic analysis of transverse dispersion in density‐coupled transport in aquifers , 2003 .

[10]  Michael N. Gooseff,et al.  Sensitivity analysis of conservative and reactive stream transient storage models applied to field data from multiple-reach experiments , 2005 .

[11]  Mary C. Hill,et al.  Predictive modeling of flow and transport in a two‐dimensional intermediate‐scale, heterogeneous porous medium , 2001 .

[12]  Ming Ye,et al.  Maximum likelihood Bayesian averaging of spatial variability models in unsaturated fractured tuff , 2003 .

[13]  J. Doherty,et al.  Role of the calibration process in reducing model predictive error , 2005 .

[14]  Stefano Tarantola,et al.  Estimating the approximation error when fixing unessential factors in global sensitivity analysis , 2007, Reliab. Eng. Syst. Saf..

[15]  L Foglia,et al.  Testing Alternative Ground Water Models Using Cross‐Validation and Other Methods , 2007, Ground water.

[16]  David Anderson,et al.  Multimodel Ranking and Inference in Ground Water Modeling , 2004, Ground water.

[17]  Michael N Fienen,et al.  On Constraining Pilot Point Calibration with Regularization in PEST , 2009, Ground water.

[18]  D. M. Ely,et al.  Preliminary evaluation of the importance of existing hydraulic-head observation locations to advective-transport predictions, Death Valley regional flow system, California and Nevada , 2001 .

[19]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[20]  Mary C. Hill,et al.  Two-dimensional advective transport in ground-water flow parameter estimation , 1996 .

[21]  J. Kirchner.  Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology , 2006 .

[22]  Mary C. Hill,et al.  Methods and Guidelines for Effective Model Calibration , 2000 .

[23]  Mary C Hill,et al.  The Practical Use of Simplicity in Developing Ground Water Models , 2006, Ground water.

[24]  C. Tiedeman,et al.  Methods for using groundwater model predictions to guide hydrogeologic data collection, with application to the Death Valley regional groundwater flow system , 2003 .

[25]  Heidi Christiansen Barlebo,et al.  Concentration data and dimensionality in groundwater models: evaluation using inverse modelling , 1998 .

[26]  E. Poeter,et al.  Inverse Models: A Necessary Next Step in Ground‐Water Modeling , 1997 .

[27]  Mary C. Hill,et al.  Death valley regional ground-water flow model calibration using optimal parameter estimation methods and geoscientific information systems , 1999 .

[28]  Heidi Christiansen Barlebo,et al.  Investigating the Macrodispersion Experiment (MADE) site in Columbus, Mississippi, using a three‐dimensional inverse flow and transport model , 2004 .

[29]  S. P. Neuman,et al.  Sensitivity analysis and assessment of prior model probabilities in MLBMA with application to unsaturated fractured tuff , 2005 .

[30]  John Doherty,et al.  Two statistics for evaluating parameter identifiability and error reduction , 2009 .

[31]  M. Hill.  A computer program (MODFLOWP) for estimating parameters of a transient, three-dimensional ground-water flow model using nonlinear regression , 1992 .

[32]  Mary C Hill,et al.  Parameter and observation importance in modelling virus transport in saturated porous media-investigations in a homogenous system. , 2005, Journal of contaminant hydrology.

[33]  C. Tiedeman,et al.  Effective Groundwater Model Calibration: With Analysis of Data, Sensitivities, Predictions, and Uncertainty , 2007 .

[34]  D. M. Ely,et al.  A method for evaluating the importance of system state observations to model predictions, with application to the Death Valley regional groundwater flow system , 2004 .

[35]  M. Hill,et al.  A Comparison of Solute‐Transport Solution Techniques and Their Effect on Sensitivity Analysis and Inverse Modeling Results , 2001, Ground water.

[36]  Arlen W. Harbaugh,et al.  MODFLOW-2000, the U.S. Geological Survey modular ground-water model; user guide to the observation, sensitivity, and parameter-estimation processes and three post-processing programs , 2000 .