Output validation is a major aspect of simulation validation in general. In continu ous simulation (e.g., System Dynamics), output validation involves demonstrating that the model is able to reproduce the dynamic time patterns that have been observed in the behavior of the real system. But standard sta tistical tests cannot be readily used to make such dynamic time pattern com parisons (i.e. periodicities, amplitudes, trends...) In this paper, we propose an output validity test that addresses the above need. The test consists of comparing the autocorrelation functions of the observed and model-generated outputs. Using simulation experiments, we demonstrate the application of the test to different situations, such as systems with large noise, observation errors, and models with parameter errors. The test is shown to detect significant errors in the fundamental periods of the observed and model-generated outputs. The test can also determine if the observed output has high frequency noise components (such as observa tion errors) not present in the model output. Experiments also demonstrate that interpreting the results of the autocorrelation function test is rather intuitive and simple, which is a major advantage of the test.
[1]
George S. Fishman,et al.
The Analysis of Simulation-Generated Time Series
,
1967
.
[2]
James A. Fitzsimmons.
The use of spectral analysis to validate peanning models
,
1974
.
[3]
Gwilym M. Jenkins,et al.
Time series analysis, forecasting and control
,
1971
.
[4]
T. Wonnacott,et al.
SPECTRAL ANALYSIS OF DATA GENERATED BY SIMULATION EXPERIMENTS WITH ECONOMETRIC MODELS
,
1969
.
[5]
Yaman Barlas,et al.
Validation of system dynamics models with a sequential procedure involving multiple quantitative methods
,
1985
.
[6]
D. A. Hsu,et al.
Analysis of Simulation-Generated Responses using Autoregressive Models
,
1977
.
[7]
M. Bartlett.
On the Theoretical Specification and Sampling Properties of Autocorrelated Time‐Series
,
1946
.
[8]
Y. Barlas.
Multiple tests for validation of system dynamics type of simulation models
,
1989
.