Robust Subspace Tracking With Missing Entries: The Set-Theoretic Approach

In this paper, an Adaptive Projected Subgradient Method (APSM) based algorithm for robust subspace tracking is introduced. A properly chosen cost function is constructed at each time instance and the goal is to seek for points, which belong to the zero level set of this function; i.e., the set of points which score a zero loss. At each iteration, an outlier detection mechanism is employed, in order to conclude whether the current data vector contains outlier noise or not. In the sequel, a sparsity-promoting greedy algorithm is employed for the outlier vector estimation allowing the purification of the corrupted data from the outlier noise, prior to any further processing. Furthermore, the case where the observation vectors are partially observed is attacked via a prediction procedure, which estimates the values of the unobserved (missing) coefficients. A theoretical analysis is carried out and the simulation experiments, within the contexts of robust subspace estimation and robust matrix completion, demonstrate the enhanced performance of the proposed scheme compared to recently developed state of the art algorithms.

[1]  Sergios Theodoridis,et al.  Robust linear regression analysis - The greedy way , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[2]  Isao Yamada,et al.  The Adaptive Projected Subgradient Method Constrained by Families of Quasi-nonexpansive Mappings and Its Application to Online Learning , 2010, SIAM J. Optim..

[3]  Martin Kleinsteuber,et al.  pROST: a smoothed $$\ell _p$$ℓp-norm robust online subspace tracking method for background subtraction in video , 2013, Machine Vision and Applications.

[4]  Paulo Sergio Ramirez,et al.  Fundamentals of Adaptive Filtering , 2002 .

[5]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[6]  Sergios Theodoridis,et al.  Machine Learning: A Bayesian and Optimization Perspective , 2015 .

[7]  I. Yamada,et al.  Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions , 2005 .

[8]  Simon Haykin,et al.  Adaptive Signal Processing: Next Generation Solutions , 2010 .

[9]  Sergios Theodoridis,et al.  Sparsity-Aware Learning and Compressed Sensing: An Overview , 2012, ArXiv.

[10]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[11]  A. Robert Calderbank,et al.  PETRELS: Subspace estimation and tracking from partial observations , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Christos G. Tsinos,et al.  Sparse subspace tracking techniques for adaptive blind channel identification in OFDM systems , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  John C. S. Lui,et al.  Online Robust Subspace Tracking from Partial Information , 2011, ArXiv.

[14]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[15]  Sergios Theodoridis,et al.  Robust Linear Regression Analysis— A Greedy Approach , 2014, IEEE Transactions on Signal Processing.

[16]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[17]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.

[18]  Yongyi Yang,et al.  Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics , 1998 .

[19]  Martin Kleinsteuber,et al.  Robust PCA and subspace tracking from incomplete observations using $$\ell _0$$ℓ0-surrogates , 2012, Comput. Stat..

[20]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[21]  Shahrokh Valaee,et al.  Distributed source localization using ESPRIT algorithm , 2001, IEEE Trans. Signal Process..

[22]  Gonzalo Mateos,et al.  Robust PCA as Bilinear Decomposition With Outlier-Sparsity Regularization , 2011, IEEE Transactions on Signal Processing.

[23]  Ian T. Foster,et al.  Mapping the Gnutella Network: Properties of Large-Scale Peer-to-Peer Systems and Implications for System Design , 2002, ArXiv.

[24]  Morteza Mardani,et al.  Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors , 2014, IEEE Transactions on Signal Processing.

[25]  Sergios Theodoridis,et al.  Adaptive Multiregression in Reproducing Kernel Hilbert Spaces: The Multiaccess MIMO Channel Case , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[27]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[28]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[29]  I. Yamada,et al.  The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings , 2006 .

[30]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[31]  Morteza Mardani,et al.  Rank minimization for subspace tracking from incomplete data , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[33]  P. L. Combettes The foundations of set theoretic estimation , 1993 .

[34]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.