What to Expect When You Are Expecting on the Grassmannian

Consider an incoming sequence of vectors, all belonging to an unknown subspace <inline-formula> <tex-math notation="LaTeX">$\text{S}$</tex-math></inline-formula>, and each with many missing entries. In order to estimate <inline-formula><tex-math notation="LaTeX">$\text{S}$</tex-math></inline-formula>, it is common to partition the data into blocks and iteratively update the estimate of <inline-formula><tex-math notation="LaTeX">$\text{S}$ </tex-math></inline-formula> with each new incoming measurement block. In this letter, we investigate a rather basic question: Is it possible to identify <inline-formula><tex-math notation="LaTeX">$\text{S}$</tex-math></inline-formula> by averaging the range of the partially observed incoming measurement blocks on the Grassmannian? We show that, in general, the span of the incoming blocks is in fact a biased estimator of <inline-formula><tex-math notation="LaTeX"> $\text{S}$</tex-math></inline-formula> when data suffer from erasures, and we find an upper bound for this bias. We reach this conclusion by examining the defining optimization program for the <italic>Fréchet expectation </italic> on the Grassmannian, and with the aid of a sharp perturbation bound and standard large deviation results.

[1]  Huan Xu,et al.  Streaming Sparse Principal Component Analysis , 2015, ICML.

[2]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[3]  Rudrasis Chakraborty,et al.  Recursive Fréchet Mean Computation on the Grassmannian and Its Applications to Computer Vision , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Yudong Chen,et al.  Incoherence-Optimal Matrix Completion , 2013, IEEE Transactions on Information Theory.

[5]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[6]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[7]  Manfred K. Warmuth,et al.  Randomized Online PCA Algorithms with Regret Bounds that are Logarithmic in the Dimension , 2008 .

[8]  L. Tong,et al.  Multichannel blind identification: from subspace to maximum likelihood methods , 1998, Proc. IEEE.

[9]  Søren Hauberg,et al.  Intrinsic Grassmann Averages for Online Linear and Robust Subspace Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Iwao Kanno,et al.  Activation detection in functional MRI using subspace modeling and maximum likelihood estimation , 1999, IEEE Transactions on Medical Imaging.

[11]  Rebecca Willett,et al.  Matrix Completion Under Monotonic Single Index Models , 2015, NIPS.

[12]  S. Chatterjee,et al.  Matrix estimation by Universal Singular Value Thresholding , 2012, 1212.1247.

[13]  M. Fréchet.  Les éléments aléatoires de nature quelconque dans un espace distancié , 1948 .

[14]  Stephen J. Wright,et al.  Local Convergence of an Algorithm for Subspace Identification from Partial Data , 2013, Found. Comput. Math..

[15]  P. Absil,et al.  Riemannian Geometry of Grassmann Manifolds with a View on Algorithmic Computation , 2004 .

[16]  Bart De Moor,et al.  Subspace Identification for Linear Systems: Theory ― Implementation ― Applications , 2011 .

[17]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[18]  Nathan Srebro,et al.  Stochastic optimization for PCA and PLS , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[19]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[20]  Rudrasis Chakraborty,et al.  An efficient recursive estimator of the Fréchet mean on a hypersphere with applications to Medical Image Analysis , 2015 .

[21]  Ioannis Mitliagkas.  Streaming PCA with Many Missing Entries , 2014 .

[22]  H. Karcher.  Riemannian center of mass and mollifier smoothing , 1977 .

[23]  Martin Styner,et al.  Regression Uncertainty on the Grassmannian , 2017, AISTATS.

[24]  Dimitris G. Manolakis,et al.  Detection algorithms for hyperspectral imaging applications , 2002, IEEE Signal Process. Mag..

[25]  P. Wedin.  Perturbation bounds in connection with singular value decomposition , 1972 .

[26]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[27]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[28]  I. Johnstone,et al.  On Consistency and Sparsity for Principal Components Analysis in High Dimensions , 2009, Journal of the American Statistical Association.

[29]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[30]  Dehui Yang,et al.  SNIPE for Memory-Limited PCA From Incomplete Data , 2016, ArXiv.

[31]  H. P..  Annales de l'Institut Henri Poincaré , 1931, Nature.