Statistical validation of simulation models

This paper investigates various statistical methodologies for validating simulation models in automotive design. Validation metrics to compare model prediction with experimental observation, when there is uncertainty in both, are developed. Two types of metrics based on Bayesian analysis and principal components analysis are proposed. The validation results are also compared with those obtained from classical hypothesis testing. A fatigue life prediction model for composite materials and a residual stress prediction model for a spot-welded joint are validated, using the proposed methodology.

[1]  Pia Veldt Larsen,et al.  In All Likelihood: Statistical Modelling and Inference Using Likelihood , 2003 .

[2]  J. Berger,et al.  The Intrinsic Bayes Factor for Model Selection and Prediction , 1996 .

[3]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[4]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[5]  J. Crawford.  A Toolbox for Validation of Nonlinear Finite Element Models , 2022 .

[6]  Wilson H. Tang,et al.  Probability concepts in engineering planning and design , 1984 .

[7]  J. Berger,et al.  Testing Precise Hypotheses , 1987 .

[8]  Dale E. Seborg,et al.  Matching patterns from historical data using PCA and distance similarity factors , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[9]  Sankaran Mahadevan,et al.  Bayesian methodology for reliability model acceptance , 2003, Reliab. Eng. Syst. Saf..

[10]  David E. Booth,et al.  Applied Multivariate Analysis , 2003, Technometrics.

[11]  Timothy G. Trucano,et al.  Statistical Validation of Engineering and Scientific Models: Background , 1999 .

[12]  A. Rukhin.  Bayes and Empirical Bayes Methods for Data Analysis , 1997 .

[13]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[14]  Sankaran Mahadevan,et al.  Probabilistic fatigue life prediction of multidirectional composite laminates , 2005 .

[15]  Timothy G. Trucano,et al.  Verification and Validation in Computational Fluid Dynamics , 2002 .

[16]  Douglas H. Johnson.  The Insignificance of Statistical Significance Testing , 1999 .

[17]  M. B. Beck,et al.  Water quality modeling: A review of the analysis of uncertainty , 1987 .

[18]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[19]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[20]  Nagui M. Rouphail,et al.  STATISTICALLY-BASED VALIDATION OF COMPUTER SIMULATION MODELS IN TRAFFIC OPERATIONS AND MANAGEMENT , 2002 .

[21]  James O. Berger,et al.  A Framework for Validation of Computer Models , 2007, Technometrics.

[22]  Shin Ishii,et al.  A Bayesian missing value estimation method for gene expression profile data , 2003, Bioinform..

[23]  Joydeep Ghosh,et al.  Relationship-based clustering and cluster ensembles for high-dimensional data mining , 2002 .

[24]  Darren T. Andrews,et al.  Maximum likelihood principal component analysis , 1997 .

[25]  John S. J. Hsu,et al.  Bayesian Methods: An Analysis for Statisticians and Interdisciplinary Researchers , 1999 .

[26]  T. G. Trucano,et al.  Confidence in ASCI scientific simulations , 1998 .

[27]  Ian H. Leslie,et al.  Statistical Validation of Engineering and Scientific Models: Validation Experiments to Application , 2003 .

[28]  William L. Oberkampf,et al.  Guide for the verification and validation of computational fluid dynamics simulations , 1998 .

[29]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[30]  Sankaran Mahadevan,et al.  Validation and error estimation of computational models , 2006, Reliab. Eng. Syst. Saf..

[31]  Christian P. Robert,et al.  Estimation of Accuracy in Testing , 1992 .

[32]  C. F. Tsang,et al.  A Broad View of Model Validation , 1989 .

[33]  D. H. Bae,et al.  Assessing the effects of residual stresses on the fatigue strength of spot welds: Residual stress at the nugget's edge was taken into consideration in the evaluation of fatigue strength for spot welds of various shapes and dimensions , 2003 .

[34]  Alfredo Ang H.-S.,et al.  Probability concepts in engineering planning and design, vol i : basic principles , 1979 .

[35]  J. D. Chrostowski,et al.  Effects of product and experimental variability on model verification of automobile structures , 1997 .

[36]  Keng C. Yap,et al.  Assessment of model accuracy relative to stochastic system behavior. , 2003 .

[37]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[38]  William L. Oberkampf,et al.  Experimental methodology for computational fluid dynamics code validation , 1997 .