A Comprehensive Metric for Comparing Time Histories in Validation of Simulation Models With Emphasis on Vehicle Safety Applications

Computer modeling and simulation are the cornerstones of product design and development in the automotive industry. Computer-aided engineering tools have improved to the extent that virtual testing may lead to significant reduction in prototype building and testing of vehicle designs. In order to make this a reality, we need to assess our confidence in the predictive capabilities of simulation models. As a first step in this direction, this paper deals with developing a metric to compare time histories that are outputs of simulation models to time histories from experimental tests with emphasis on vehicle safety applications. We focus on quantifying discrepancy between time histories as the latter constitute the predominant form of responses of interest in vehicle safety considerations. First we evaluate popular measures used to quantify discrepancy between time histories in fields such as statistics, computational mechanics, signal processing, and data mining. Then we propose a structured combination of some of these measures and define a comprehensive metric that encapsulates the important aspects of time history comparison. The new metric classifies error components associated with three physically meaningful characteristics (phase, magnitude and topology), and utilizes norms, cross-correlation measures and algorithms such as dynamic time warping to quantify discrepancies. Two case studies demonstrate that the proposed metric seems to be more consistent than existing metrics. It is also shown how the metric can be used in conjunction with ratings from subject matter experts to build regression-based validation models.Copyright © 2008 by ASME

[1]  Yan Fu,et al.  Development of Validation Metrics for Vehicle Frontal Impact Simulation , 2007, DAC 2007.

[2]  Ren-Jye Yang,et al.  CAE Model Validation in Vehicle Safety Design , 2004 .

[3]  Pietro Perona,et al.  Visual Identification by Signature Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Richard M. Morgan,et al.  Durability, Repeatability and Reproducibility of the NHTSA Side Impact Dummy , 1983 .

[5]  Poong Hyun Seong,et al.  A signal pattern matching and verification method using interval means cross correlation and eigenvalues in the nuclear power plant monitoring systems , 2002 .

[6]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[7]  William L. Oberkampf,et al.  Guide for the verification and validation of computational fluid dynamics simulations , 1998 .

[8]  Xavier Trosseille,et al.  Mathematical models integral rating , 2000 .

[9]  Francisco Casacuberta,et al.  Is the DTW "distance" really a metric? An algorithm reducing the number of DTW comparisons in isolated word recognition , 1985, Speech Commun..

[10]  Weigang Chen,et al.  Automated Occupant Model Evaluation and Correlation , 2005 .

[11]  Yang Li,et al.  Synchronization of batch trajectory based on multi-scale dynamic time warping , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[12]  Suresh Venkatasubramanian,et al.  Curve Matching, Time Warping, and Light Fields: New Algorithms for Computing Similarity between Curves , 2007, Journal of Mathematical Imaging and Vision.

[13]  Leonard E. Schwer Validation Metrics for Response Histories: A Review with Case Studies , 2006 .

[14]  Michael A. Sprague,et al.  A spectral‐element method for modelling cavitation in transient fluid–structure interaction , 2004 .

[15]  Paul R. Cohen,et al.  Using Dynamic Time Warping to Bootstrap HMM-Based Clustering of Time Series , 2001, Sequence Learning.

[16]  Paolo Ciaccia,et al.  Warping the Time on Data Streams , 2005, SBBD.

[17]  Matthew F. Barone,et al.  Measures of agreement between computation and experiment: Validation metrics , 2004, J. Comput. Phys..

[18]  Clement T. Yu,et al.  Haar Wavelets for Efficient Similarity Search of Time-Series: With and Without Time Warping , 2003, IEEE Trans. Knowl. Data Eng..

[19]  David M. Russell Error Measures for Comparing Transient Data : Part I : Development of a Comprehensive Error Measure Part II : Error Measures Case Study , 2000 .

[20]  Marcos Faúndez-Zanuy,et al.  On-line signature recognition based on VQ-DTW , 2007, Pattern Recognit..

[21]  Esther M. Arkin,et al.  An efficiently computable metric for comparing polygonal shapes , 1991, SODA '90.