Defining Coverage of a Domain Using a Modified Nearest-Neighbor Metric

Validation experiments are conducted at discrete settings within the domain of interest to assess the predictive maturity of a model over the entire domain. Satisfactory model performance merely at these discrete tested settings is insufficient to ensure that the model will perform well throughout the domain, particularly at settings far from validation experiments. The goal of coverage metrics is to reveal how well a set of validation experiments represents the entire operational domain. The authors identify the criteria of an exemplary coverage metric, evaluate the ability of existing coverage metrics to fulfill each criterion, and propose a new, improved coverage metric. The proposed metric favors interpolation over extrapolation through a penalty function, causing the metric to prefer a design of validation experiments near the boundaries of the domain, while simultaneously exploring inside the domain. Furthermore, the proposed metric allows the coverage to account for uncertainty associated with validation experiments. Application of the proposed coverage metric on a practical, non-trivial problem is demonstrated on the Viscoplastic Self-Consistent material plasticity code for 5182 aluminum alloy.