Calibrating and assessing uncertainty in coastal numerical models

Abstract Advanced numerical models used to predict coastal change at a variety of time and spatial scales often contain many free parameters that require calibration to the available field data. At present, little guidance (beyond the adoption of the default values provided) is available in the field of coastal engineering to inform the selection of best-fit parameter values. Common calibration techniques can often lack a rigorous quantification of model sensitivity to parameters and parameter-induced model uncertainty. Here we employ the Generalised Likelihood Uncertainty Estimation (GLUE) method to address these issues. The GLUE method uses Monte Carlo sampling to assess the skill of many different combinations of model parameters when compared to observational data. As a rigorous modelling framework, the GLUE method provides a series of standard tools that assist the modeller to analyse model sensitivity, undertake parameter optimisation and quantify parameter-induced uncertainty. In addition, new tools are presented here to identify where unique calibrated parameter sets are required for different observational data (e.g., should the calibrated parameter set differ between alongshore locations at a site) and investigate the convergence of GLUE estimated optimum parameter values over increasing numbers of Monte Carlo samples. As the methodology and philosophy of GLUE is well established in other fields, this paper presents a practical case study to explore the strengths and weaknesses of the method when applied to a relatively complex coastal numerical model (XBeach). The results obtained are compared to a previously reported and more ‘standard’ model calibration undertaken within the context of a coastal storm early warning system. While the GLUE method requires orders of magnitude more computational power, it is shown that its use in place of the more common one-at-a-time ‘trial-and-error’ approach to model calibration, provides: a significant improvement in predictive skill; a more rigorous evaluation of the model sensitivity to parameters; the ability to identify distinct differences in the XBeach model performance dependent on dune impact processes; and additional analysis including the quantification of parameter-induced uncertainty.

[1]  Richard Soulsby,et al.  Evaluating the performance of morphological models , 2004 .

[2]  Keith Beven,et al.  The future of distributed models: model calibration and uncertainty prediction. , 1992 .

[3]  P. Ciavola,et al.  Dynamics of a nearshore bar system in the northern Adriatic: A video-based morphological classification , 2011 .

[4]  K. Beven,et al.  Equifinality and uncertainty in physically based soil erosion models: Application of the glue methodology to WEPP-the water erosion prediction project-for sites in the UK and USA , 2000 .

[5]  Nathaniel G. Plant,et al.  Prediction skill of nearshore profile evolution models , 2004 .

[6]  Nathaniel G. Plant,et al.  Prediction and assimilation of surf-zone processes using a Bayesian network , 2011 .

[7]  Robert McCall,et al.  Modelling storm hydrodynamics on gravel beaches with XBeach-G , 2014 .

[8]  B. G. Ruessink A Bayesian estimation of parameter-induced uncertainty in a nearshore alongshore current model , 2006 .

[9]  H. Karunarathna,et al.  A statistical-process based approach for modelling beach profile variability , 2013 .

[10]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[11]  J. A. Roelvink,et al.  Dissipation in random wave groups incident on a beach , 1993 .

[12]  B. G. Ruessink,et al.  Calibration of nearshore process models—application of a hybrid genetic algorithm , 2005 .

[13]  Steve Elgar,et al.  Modeling the alongshore current on barred beaches , 2001 .

[14]  Qi Zhang,et al.  Parameter and modeling uncertainty simulated by GLUE and a formal Bayesian method for a conceptual hydrological model , 2010 .

[15]  David P. Callaghan,et al.  Statistical simulation of wave climate and extreme beach erosion , 2008 .

[16]  Gerben Ruessink,et al.  Process-based modeling of cross-shore sandbar behavior , 2015 .

[17]  B. G. Ruessink,et al.  Predictive uncertainty of a nearshore bed evolution model , 2005 .

[18]  S. Elgar,et al.  Wave-induced sediment transport and onshore sandbar migration , 2006 .

[19]  Keith Beven,et al.  Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology , 2001 .

[20]  Paolo Ciavola,et al.  Storm impacts along European coastlines. Part 1: The joint effort of the MICORE and ConHaz Projects , 2011 .

[21]  Margaret L. Palmsten,et al.  Laboratory investigation of dune erosion using stereo video , 2012 .

[22]  L. Wright,et al.  Morphodynamic variability of surf zones and beaches: A synthesis , 1984 .

[23]  N. Plant,et al.  Two-dimensional time dependent hurricane overwash and erosion modeling at Santa Rosa Island , 2010 .

[24]  Demetris Koutsoyiannis,et al.  One decade of multi-objective calibration approaches in hydrological modelling: a review , 2010 .

[25]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .

[26]  Kristen D. Splinter,et al.  Modeling dune response to an East Coast Low , 2012 .

[27]  Søren Liedtke Thorndahl,et al.  Event based uncertainty assessment in urban drainage modelling applying the GLUE methodology , 2008 .

[28]  Kristen D. Splinter,et al.  Observations and simulations of wave runup during a laboratory dune erosion experiment , 2016 .

[29]  M. Larson,et al.  An analytical model to predict dune erosion due to wave impact , 2004 .

[30]  David P. Callaghan,et al.  Probabilistic modelling of extreme storms along the Dutch coast , 2014 .

[31]  N. Plant,et al.  Modelling multi-hazard hurricane damages on an urbanized coast with a Bayesian Network approach , 2015 .

[32]  K. Beven,et al.  Bayesian Estimation of Uncertainty in Runoff Prediction and the Value of Data: An Application of the GLUE Approach , 1996 .

[33]  Leonardo Noto,et al.  Influence of surface roughness in hydrological response of semiarid catchments , 2005 .

[34]  David P. Callaghan,et al.  Probabilistic estimation of storm erosion using analytical, semi-empirical, and process based storm erosion models , 2013 .

[35]  P. van Gelder,et al.  Probabilistic modeling of wave climate and predicting dune erosion , 2013 .

[36]  Luis Pedro Almeida,et al.  Toward reliable storm-hazard forecasts: XBeach calibration and its potential application in an operational early-warning system , 2012, Ocean Dynamics.

[37]  Marcel J. F. Stive,et al.  Modelling dune erosion, overwash and breaching at Fire Island (NY) during hurricane Sandy , 2015 .

[38]  James W. Jones,et al.  Influence of likelihood function choice for estimating crop model parameters using the generalized likelihood uncertainty estimation method , 2010 .

[39]  Shunqi Pan,et al.  The predictability of cross-shore bed evolution of sandy beaches at the time scale of storms and seasons using process-based Profile models , 2003 .

[40]  Mitchell D. Harley,et al.  Can an early warning system help minimize the impacts of coastal storms? A case study of the 2012 Halloween storm, Northern Italy , 2015 .

[41]  P. Mantovan,et al.  Hydrological forecasting uncertainty assessment: Incoherence of the GLUE methodology , 2006 .

[42]  Peter N. Adams,et al.  Development of the Coastal Storm Modeling System (CoSMoS) for predicting the impact of storms on high-energy, active-margin coasts , 2014, Natural Hazards.

[43]  Jr. Asbury H. Sallenger Storm Impact Scale for Barrier Islands , 2000 .

[44]  D. Roelvink,et al.  Modelling storm impacts on beaches, dunes and barrier islands , 2009 .

[45]  Nathaniel G. Plant,et al.  Probabilistic prediction of barrier‐island response to hurricanes , 2012 .

[46]  G. Hornberger,et al.  Approach to the preliminary analysis of environmental systems , 1981 .

[47]  Paolo Ciavola,et al.  Critical storm thresholds for significant morphological changes and damage along the Emilia-Romagna coastline, Italy , 2012 .

[48]  Ad Reniers,et al.  Modeling cross-shore sandbar behavior on the timescale of weeks , 2007 .

[49]  Nathaniel G. Plant,et al.  Evaluation of wave runup predictions from numerical and parametric models , 2014 .

[50]  Andrew Binley,et al.  GLUE: 20 years on , 2014 .

[51]  Ryan J. Lowe,et al.  Numerical modeling of low-frequency wave dynamics over a fringing coral reef , 2013 .

[52]  R. Spear Eutrophication in peel inlet—II. Identification of critical uncertainties via generalized sensitivity analysis , 1980 .

[53]  Ap van Dongeren,et al.  Predicting coastal hazards for sandy coasts with a Bayesian Network , 2016 .

[54]  J. Stedinger,et al.  Appraisal of the generalized likelihood uncertainty estimation (GLUE) method , 2008 .

[55]  Mitchell D. Harley,et al.  Evaluation of XBeach predictions for a real-time warning system in Emilia-Romagna, Northern Italy , 2011 .

[56]  Robert G. Dean,et al.  Convolution Method for Time-Dependent Beach-Profile Response , 1993 .