A Simulation-Based Method to Determine Model Misspecification: Examples using Natural Mortality and Population Dynamics Models

Abstract Recent developments in the models used in wildlife and fisheries science have allowed the inclusion of a wider range of data than previously. However, the diagnostics of such complex models have not kept pace. We describe a new diagnostic technique based on simulation analysis. Model misspecification was identified through simulation methods that created a distribution of likely parameter values for a model that was correctly specified. If the actual estimate of that parameter is outside the bounds of the simulated distribution, then the model is probably misspecified. We tested the reliability of the new diagnostic by introducing known-model misspecification into complex fisheries stock assessment models. We then compared the results from this new diagnostic with those of a more tradition fisheries diagnostic, namely, retrospective analysis. The simulation-based diagnostic was shown to identify inisspecification affecting the estimated dynamics more reliably than retrospective analysis.

[1]  T. E. Helser,et al.  Growth of silver hake within the U.S. continental shelf ecosystem of the northwest Atlantic Ocean , 1996 .

[2]  D. Cox,et al.  A General Definition of Residuals , 1968 .

[3]  Jerry Nedelman,et al.  Book review: “Bayesian Data Analysis,” Second Edition by A. Gelman, J.B. Carlin, H.S. Stern, and D.B. Rubin Chapman & Hall/CRC, 2004 , 2005, Comput. Stat..

[4]  Rowan Haigh,et al.  Compositional analysis of catch curve data, with an application to Sebastes maliger , 2007 .

[5]  H. Skaug,et al.  Automatic Approximation of the Marginal Likelihood in Nonlinear Hierarchical Models , 2004 .

[6]  Hans J. Skaug,et al.  Automatic approximation of the marginal likelihood in non-Gaussian hierarchical models , 2006, Comput. Stat. Data Anal..

[7]  Xiao-Li Meng,et al.  Posterior Predictive $p$-Values , 1994 .

[8]  Andrew R. Solow,et al.  A simple test for change in community structure , 1993 .

[9]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[10]  John Sibert,et al.  AD Model Builder: using automatic differentiation for statistical inference of highly parameterized complex nonlinear models , 2012, Optim. Methods Softw..

[11]  M. Maunder,et al.  Implications of model and data assumptions: An illustration including data for the Taiwanese longline fishery into the eastern Pacific Ocean bigeye tuna (Thunnus obesus) stock assessment , 2009 .

[12]  Scott M. Lynch,et al.  Bayesian Posterior Predictive Checks for Complex Models , 2004 .

[13]  R. Hilborn,et al.  Fisheries stock assessment and decision analysis: the Bayesian approach , 1997, Reviews in Fish Biology and Fisheries.

[14]  J. Ianelli,et al.  Designing tools to evaluate fishery management strategies: can the scientific community deliver? , 2007 .

[15]  N. G. Elliott,et al.  Global population structure of yellowfin tuna, Thunnus albacares, inferred from allozyme and mitochondrial DNA variation , 1997 .

[16]  D. Rubin.  Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician , 1984 .

[17]  James N. Ianelli,et al.  Computers in Fisheries Population Dynamics , 2009 .

[18]  M. Niknian,et al.  Randomization and Monte Carlo Methods in Biology , 1993 .

[19]  David R. Anderson,et al.  Suggestions for presenting the results of data analyses , 2001 .

[20]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[21]  D. Adams,et al.  Using randomization techniques to analyse behavioural data , 1996, Animal Behaviour.

[22]  Patrick J. Farrell,et al.  Local influence diagnostics for the retrospective problem in sequential population analysis , 2005 .