Robustness, Fidelity and Prediction-Looseness of Models

Assessment of the credibility of a mathematical or numerical model of a complex system must combine three components: (1) The fidelity of the model to test data, e.g. as quantified by a mean squared error. (2) The robustness, of model fidelity, to lack of understanding of the underlying processes. (3) The prediction looseness of the model. ‘Prediction looseness’ is the range of predictions of models that are equivalent in terms of fidelity. The main result of this paper asserts that fidelity, robustness, and prediction looseness are mutually antagonistic. A change in the model that enhances one of these attributes will cause deterioration of another. In particular, increasing the delity to test data will decrease the robustness to imperfect understanding of the process. Likewise, increasing the robustness will increase the predictive looseness. The conclusion is that focusing only on fidelity-to-data is not a sound decision-making strategy for model building and