Online low-rank subspace learning from incomplete data using rank revealing ℓ2/ℓ1 regularization

Massive amounts of data (also called big data) generated by a wealth of sources such as social networks, satellite sensors etc., necessitate the deployment of efficient processing tools. In this context, online subspace learning algorithms that aim at retrieving low-rank representations of data constitute a mainstay in many applications. Working with incomplete (partially observed) data has recently become commonplace. Moreover, the knowledge of the real rank of the sought subspace is rarely at our disposal a priori. Herein, a novel low-rank subspace learning algorithm from incomplete data is presented. Its main premise is the online processing of incomplete data along with the imposition of low-rankness on the sought subspace via a sophisticated utilization of the group sparsity inducing ℓ2/ℓ1 norm. As is experimentally shown, the resulting scheme is efficient in accurately learning the subspace as well as in unveiling its real rank.

[1]  Gonzalo Mateos,et al.  Modeling and Optimization for Big Data Analytics: (Statistical) learning tools for our era of data deluge , 2014, IEEE Signal Processing Magazine.

[2]  Ender M. Eksioglu,et al.  Group sparse RLS algorithms , 2014 .

[3]  IEEE Statistical Signal Processing Workshop, SSP 2012, Ann Arbor, MI, USA, August 5-8, 2012 , 2012, Symposium on Software Performance.

[4]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[5]  Aggelos K. Katsaggelos,et al.  Sparse Bayesian Methods for Low-Rank Matrix Estimation , 2011, IEEE Transactions on Signal Processing.

[6]  C. Févotte,et al.  Automatic Relevance Determination in Nonnegative Matrix Factorization , 2009 .

[7]  Robert D. Nowak,et al.  Online identification and tracking of subspaces from highly incomplete information , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[8]  Paris V. Giampouras,et al.  Simultaneously Sparse and Low-Rank Abundance Matrix Estimation for Hyperspectral Image Unmixing , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Konstantinos Koutroumbas,et al.  Online Bayesian low-rank subspace learning from partial observations , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[10]  Morteza Mardani,et al.  Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors , 2014, IEEE Transactions on Signal Processing.

[11]  A. Robert Calderbank,et al.  PETRELS: Parallel Subspace Estimation and Tracking by Recursive Least Squares From Partial Observations , 2012, IEEE Transactions on Signal Processing.

[12]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[13]  Sergios Theodoridis,et al.  Machine Learning: A Bayesian and Optimization Perspective , 2015 .