Goodness-offit tests on manifolds

We develop a general theory for the goodness-offit test to non-linear models. In particular, we assume that the observations are noisy samples of a sub-manifold defined by a non-linear map of some intrinsic structures. The observation noise is additive Gaussian. Our main result shows that the “residual” of the model fit, by solving a non-linear least-square problem, follows a (possibly non-central) χ distribution. The parameters of the χ distribution are related to the model order and dimension of the problem. The main result is established by making a novel connection between statistical test and differential geometry. We further present a method to select the model orders sequentially. We demonstrate the broad application of the general theory in a range of applications in machine learning and signal processing, including determining the rank of low-rank (possibly complex-valued) matrices and tensors, from noisy, partial, or indirect observations, determining the number of sources in signal demixing, and potential applications in determining the number of hidden nodes in neural networks.

[1]  S. Sternberg.  Lectures on Differential Geometry , 1964 .

[2]  F. Fisher,et al.  The Identification Problem in Econometrics. , 1967 .

[3]  A. Shapiro,et al.  On the multivariate asymptotic distribution of sequential Chi-square statistics , 1985 .

[4]  A. Shapiro.  Asymptotic Theory of Overparameterized Structural Models , 1986 .

[5]  Douglas A. McManus.  Who Invented Local Power Analysis? , 1991, Econometric Theory.

[6]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[7]  L. Wasserman,et al.  Bayesian goodness-of-fit testing using infinite-dimensional exponential families , 1998 .

[8]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[9]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[10]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[11]  K. Wapenaar,et al.  Imaging with ambient noise , 2010 .

[12]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[13]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[14]  Joel A. Tropp,et al.  Sharp Recovery Bounds for Convex Demixing, with Applications , 2012, Found. Comput. Math..

[15]  Lieven De Lathauwer,et al.  Generic Uniqueness Conditions for the Canonical Polyadic Decomposition and INDSCAL , 2014, SIAM J. Matrix Anal. Appl..

[16]  Justin K. Romberg,et al.  An Overview of Low-Rank Matrix Recovery From Incomplete Observations , 2016, IEEE Journal of Selected Topics in Signal Processing.

[17]  Qiang Liu,et al.  A Kernelized Stein Discrepancy for Goodness-of-fit Tests , 2016, ICML.

[18]  Thomas Strohmer,et al.  Blind Deconvolution Meets Blind Demixing: Algorithms and Performance Bounds , 2015, IEEE Transactions on Information Theory.

[19]  Giorgio Ottaviani,et al.  Effective Criteria for Specific Identifiability of Tensors and Forms , 2016, SIAM J. Matrix Anal. Appl..

[20]  Yuanzhi Li,et al.  Algorithmic Regularization in Over-parameterized Matrix Recovery , 2017, ArXiv.

[21]  Jie Ding,et al.  Model Selection Techniques: An Overview , 2018, IEEE Signal Processing Magazine.

[22]  Wen-Zhan Song,et al.  Communication Efficient Signal Detection for Distributed Ambient Noise Imaging , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.

[23]  Alexander Shapiro,et al.  Matrix Completion With Deterministic Pattern: A Geometric Perspective , 2018, IEEE Transactions on Signal Processing.