Beyond covariance realism: a new metric for uncertainty realism

In the space surveillance tracking domain, it is often necessary to assess not only the covariance consistency or covariance realism of an object's state estimate, but also the realism (proper characterization) of its full estimated probability density function. In other words, there is a need for “uncertainty realism." We propose a new metric (applicable to any tracking domain) that generalizes the covariance realism metric based on the Mahalanobis distance to one that tests uncertainty realism. We then review various goodness-of-fit and distribution matching tests that exploit the uncertainty realism metric and describe how these tests can be applied to assess uncertainty realism in off-line simulations with multiple Monte-Carlo trials or on-line with real data when truth is available.

[1]  H. Sorenson,et al.  Nonlinear Bayesian estimation using Gaussian sum approximations , 1972 .

[2]  Kazufumi Ito,et al.  Gaussian filters for nonlinear filtering problems , 2000, IEEE Trans. Autom. Control..

[3]  Ralph B. D'Agostino,et al.  Tests for Departure from Normality , 1973 .

[4]  Aubrey B. Poore,et al.  Gaussian Sum Filters for Space Surveillance: Theory and Simulations , 2011 .

[5]  Aubrey B. Poore,et al.  Nonlinear Uncertainty Propagation in Orbital Elements and Transformation to Cartesian Space Without Loss of Realism , 2014 .

[6]  M. Stephens.  EDF Statistics for Goodness of Fit and Some Comparisons , 1974 .

[7]  Oliver E. Drummond,et al.  Metrics for evaluating track covariance consistency , 2007, SPIE Optical Engineering + Applications.

[8]  Aubrey B. Poore,et al.  Implicit-Runge–Kutta-based methods for fast, precise, and scalable uncertainty propagation , 2015 .

[9]  Ralph B. D'Agostino,et al.  Goodness-of-Fit-Techniques , 2020 .

[10]  T. Singh,et al.  Uncertainty Propagation for Nonlinear Dynamic Systems Using Gaussian Mixture Models , 2008 .

[11]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[12]  W. J. Langford.  Statistical Methods , 1959, Nature.

[13]  Aubrey B. Poore,et al.  Orbit and uncertainty propagation: a comparison of Gauss–Legendre-, Dormand–Prince-, and Chebyshev–Picard-based approaches , 2014 .

[14]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..

[15]  James Durbin,et al.  Components of Cramer-von Mises statistics. I , 1972 .

[16]  Aubrey B. Poore,et al.  Adaptive Gaussian Sum Filters for Space Surveillance , 2011, IEEE Transactions on Automatic Control.

[17]  Gene H. Golub,et al.  Matrix computations , 1983 .

[18]  Chris Sabol,et al.  Linearized Orbit Covariance Generation and Propagation Analysis via Simple Monte Carlo Simulations (Preprint) , 2010 .

[19]  Chris Sabol,et al.  Comparison of Covariance Based Track Association Approaches Using Simulated Radar Data , 2012 .

[20]  J. W. Humberston.  Classical mechanics , 1980, Nature.

[21]  A. Wald,et al.  On the Choice of the Number of Class Intervals in the Application of the Chi Square Test , 1942 .

[22]  Aubrey B. Poore,et al.  Orbital State Uncertainty Realism , 2012 .

[23]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[24]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[25]  Aubrey B. Poore,et al.  A comparative study of new non-linear uncertainty propagation methods for space surveillance , 2014, Defense + Security Symposium.

[26]  D. Darling.  The Kolmogorov-Smirnov, Cramer-von Mises Tests , 1957 .

[27]  Julian J. Faraway,et al.  The Exact and Asymptotic Distributions of Cramer-von Mises Statistics , 1996 .

[28]  Aubrey B. Poore,et al.  Gauss von Mises Distribution for Improved Uncertainty Realism in Space Situational Awareness , 2014, SIAM/ASA J. Uncertain. Quantification.

[29]  Kyle DeMars,et al.  THE USE OF SHORT-ARC ANGLE AND ANGLE RATE DATA FOR DEEP-SPACE INITIAL ORBIT DETERMINATION AND TRACK ASSOCIATION , 2010 .