Detection of Modeling Misspecification Using Cross-Entropy Test

In many signal processing applications and learning problems, modeling misspecification may dramatically degrade the estimation performance. Thus, it is useful to detect possible misspecification in the model using the observations. In this work, a composite hypothesis test for detection of modeling misspecification is proposed in the non-Bayesian framework. The test is based on the estimated cross-entropy and thus it is called cross-entropy test (CET). It is computed by the negative of the maximized log-likelihood with respect to the unknown deterministic parameters, where the unknown deterministic parameters are substituted with their maximum likelihood estimates. It is shown that under some mild conditions, the proposed test has the constant false-alarm rate property. The CET performance is evaluated and compared via simulations to White's test for modeling misspecification detection, which is based on information matrix equality. In an example of direction-of-arrival estimation using a miscalibrated array of sensors, it is shown that unlike White's test, the CET is able to detect model misspecification.

[1]  Dennis D. Boos,et al.  The IOS Test for Model Misspecification , 2004 .

[2]  Toshiaki Koike-Akino,et al.  Misspecified CRB on Parameter Estimation for a Coupled Mixture of Polynomial Phase and Sinusoidal FM Signals , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Testing the Equality of Two Positive-Definite Matrices with Application to Information Matrix Testing , 2012 .

[4]  Alfred O. Hero,et al.  Measure-transformed quasi maximum likelihood estimation with application to source localization , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Sundeep Prabhakar Chepuri,et al.  Big data sketching with model mismatch , 2015, 2015 49th Asilomar Conference on Signals, Systems and Computers.

[6]  Fulvio Gini,et al.  The Constrained Misspecified Cramér–Rao Bound , 2016, IEEE Signal Processing Letters.

[7]  Michael Muma,et al.  Robust Estimation in Signal Processing: A Tutorial-Style Treatment of Fundamental Concepts , 2012, IEEE Signal Processing Magazine.

[8]  P. J. Huber.  The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[9]  Andrew Chesher,et al.  The information matrix test: Simplified calculation via a score test interpretation , 1983 .

[10]  Ing Rj Ser.  Approximation Theorems of Mathematical Statistics , 1980 .

[11]  Christ D. Richmond,et al.  Parameter bounds under misspecified models , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[12]  Halbert White,et al.  New Directions in Information Matrix Testing: Eigenspectrum Tests , 2013 .

[13]  Fulvio Gini,et al.  On scatter matrix estimation in the presence of unknown extra parameters: Mismatched scenario , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[14]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[15]  Fulvio Gini,et al.  Performance Bounds for Parameter Estimation under Misspecified Models: Fundamental Findings and Applications , 2017, IEEE Signal Processing Magazine.

[16]  Sean P. Meyn,et al.  Universal and Composite Hypothesis Testing via Mismatched Divergence , 2009, IEEE Transactions on Information Theory.

[17]  Peter A. Parker,et al.  Methods and bounds for waveform parameter estimation with a misspecified model , 2015, 2015 49th Asilomar Conference on Signals, Systems and Computers.

[18]  Alfred O. Hero,et al.  Robust Multiple Signal Classification via Probability Measure Transformation , 2015, IEEE Transactions on Signal Processing.

[19]  Steven S. Henley,et al.  Generalized Information Matrix Tests for Detecting Model Misspecification , 2016 .

[20]  Fulvio Gini,et al.  The Misspecified Cramer-Rao Bound and Its Application to Scatter Matrix Estimation in Complex Elliptically Symmetric Distributions , 2016, IEEE Transactions on Signal Processing.

[21]  Romain Couillet,et al.  Robust Estimates of Covariance Matrices in the Large Dimensional Regime , 2012, IEEE Transactions on Information Theory.

[22]  Koby Todros,et al.  Measure-transformed Gaussian quasi score test , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[23]  James G. M ac Kinnon.  Bootstrap Methods in Econometrics , 2006 .

[24]  Fulvio Gini,et al.  A lower bound for the Mismatched Maximum Likelihood estimator , 2015, 2015 IEEE Radar Conference (RadarCon).

[25]  Christ D. Richmond,et al.  Parameter Bounds on Estimation Accuracy Under Model Misspecification , 2015, IEEE Transactions on Signal Processing.

[26]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[27]  K. Athreya,et al.  Measure Theory and Probability Theory (Springer Texts in Statistics) , 2006 .

[28]  Christ D. Richmond,et al.  On Constraints in Parameter Estimation and Model Misspecification , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[29]  Quang Vuong,et al.  Cramer-Rao Bounds for Misspecified Models , 1986 .

[30]  Alfred O. Hero,et al.  Measure transformed quasi likelihood ratio test for Bayesian binary hypothesis testing , 2016, 2016 IEEE Statistical Signal Processing Workshop (SSP).

[31]  Ami Wiesel,et al.  A robust maximum likelihood multiuser detector in the presence of signature uncertainties , 2004, 2004 12th European Signal Processing Conference.

[32]  Alfred O. Hero,et al.  Robust measure transformed music for DOA estimation , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Koby Todros,et al.  Plug-In Measure-Transformed Quasi-Likelihood Ratio Test for Random Signal Detection , 2017, IEEE Signal Processing Letters.

[34]  Alfred O. Hero,et al.  Measure-transformed quasi likelihood ratio test , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  Tony Lancaster,et al.  THE COVARIANCE MATRIX OF THE INFORMATION MATRIX TEST , 1984 .

[36]  Jun S. Liu,et al.  Model selection principles in misspecified models , 2010, 1005.5483.

[37]  Wei Liu,et al.  Robust DOA Estimation for Sources With Known Waveforms Against Doppler Shifts via Oblique Projection , 2018, IEEE Sensors Journal.