Temporal Regularized Matrix Factorization

Matrix factorization approaches have been applied to a variety of applications, from recommendation systems to multi-label learning. Standard low rank matrix factorization methods fail in cases when the data can be modeled as a time series, since they do not take into account the dependencies among factors, while EM algorithms designed for time series data are inapplicable to large multiple time series data. To overcome this, matrix factorization approaches are augmented with dynamic linear model based regularization frameworks. A major drawback in such approaches is that the exact dependencies between the latent factors are assumed to be known. In this paper, we introduce a Temporal Regularized Matrix Factorization (TRMF) method, an efficient alternating minimization scheme that not only learns the latent time series factors, but also the dependencies among the latent factors. TRMF is highly general, and subsumes several existing matrix factorization approaches for time series data. We make interesting connections to graph based matrix factorization methods in the context of learning the dependencies. Experiments on both real and synthetic data show that TRMF is highly scalable, and outperforms several existing approaches used for common large scale time series tasks.

[1]  Geoffrey E. Hinton,et al.  Parameter estimation for linear dynamical systems , 1996 .

[2]  Martin J. Wainwright,et al.  Estimation of (near) low-rank matrices with noise and high-dimensional scaling , 2009, ICML.

[3]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[4]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[5]  A. Cichocki,et al.  Nonnegative Matrix Factorization with Temporal Smoothness and / or Spatial Decorrelation Constraints , 2005 .

[6]  Pradeep Ravikumar,et al.  Collaborative Filtering with Graph Information: Consistency and Scalable Methods , 2015, NIPS.

[7]  Stephen J. Wright,et al.  Forward–Backward Greedy Algorithms for Atomic Norm Regularization , 2014, IEEE Transactions on Signal Processing.

[8]  Miao Xu,et al.  Speedup Matrix Completion with Side Information: Application to Multi-Label Learning , 2013, NIPS.

[9]  Joydeep Ghosh,et al.  LAMORE: A Stable, Scalable Approach to Latent Vector Autoregressive Modeling of Categorical Time Series , 2014, AISTATS.

[10]  William B. Nicholson,et al.  Structured Regularization for Large Vector Autoregression , 2014 .

[11]  Kush R. Varshney,et al.  Dynamic matrix factorization: A state space approach , 2011, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Holger Rauhut,et al.  Low rank matrix recovery from rank one measurements , 2014, ArXiv.

[13]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[14]  Stephen P. Boyd,et al.  1 Trend Filtering , 2009, SIAM Rev..

[15]  Guillermo Sapiro,et al.  Kernelized Probabilistic Matrix Factorization: Exploiting Graphs and Side Information , 2012, SDM.

[16]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[17]  Christos Faloutsos,et al.  DynaMMo: mining and summarization of coevolving sequences with missing values , 2009, KDD.

[18]  Chih-Ling Tsai,et al.  Regression coefficient and autoregressive order shrinkage and selection via the lasso , 2007 .

[19]  E. Prescott,et al.  Postwar U.S. Business Cycles: An Empirical Investigation , 1997 .

[20]  Fang Han,et al.  Transition Matrix Estimation in High Dimensional Time Series , 2013, ICML.

[21]  Walter Willinger,et al.  Spatio-Temporal Compressive Sensing and Internet Traffic Matrices (Extended Version) , 2012, IEEE/ACM Transactions on Networking.

[22]  Walter Willinger,et al.  Spatio-temporal compressive sensing and internet traffic matrices , 2009, SIGCOMM '09.

[23]  R. Shumway,et al.  AN APPROACH TO TIME SERIES SMOOTHING AND FORECASTING USING THE EM ALGORITHM , 1982 .

[24]  Lei Li,et al.  Time Series Clustering: Complex is Simpler! , 2011, ICML.

[25]  Yin Zhang,et al.  Exploiting temporal stability and low-rank structure for localization in mobile networks , 2010, MobiCom.

[26]  Xi Chen,et al.  Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization , 2010, SDM.