Total variation based tensor decomposition for multi‐dimensional data with time dimension

In this paper, we study tensors with time dimension which arises in many scientific and engineering applications such as time series gene expression analysis and video analysis. In these applications, we are interested in determining a set of components interacting closely and consistently together over a period of time. The main aim of this paper is to develop a numerical method to compute such constrained CANDECOMP/PARAFAC (CP) decompositions. We make use of the total variation regularization to constrain the time dimension factor in the decomposition in order to obtain a piecewise constant function with respect to time points. The components of the other dimensions corresponding to these time points are closely related. For example, in time series gene expression analysis, a set of genes may regulate a biological process together during a specific time period; in video analysis, a set of image pixels may refer to a foreground object in the video frames. We employ ADMM to solve the resulting optimization problem, and study its convergence. Numerical examples on synthetic and real data sets are used to demonstrate that the proposed total variation based CP decomposition model can provide more accurate and interesting results. Copyright © 2015 John Wiley & Sons, Ltd.

[1]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[2]  M. Ng,et al.  Multi-view foreground segmentation via fourth order tensor learning , 2013 .

[3]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[4]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[5]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[6]  Lieven De Lathauwer,et al.  Tensor-based techniques for the blind separation of DS-CDMA signals , 2007, Signal Process..

[7]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[8]  J. Chisholm.  Approximation by Sequences of Padé Approximants in Regions of Meromorphy , 1966 .

[9]  J. Mocks,et al.  Topographic components model for event-related potentials and some biophysical considerations , 1988, IEEE Transactions on Biomedical Engineering.

[10]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .

[11]  Yunming Ye,et al.  HAR: Hub, Authority and Relevance Scores in Multi-Relational Data for Query Search , 2012, SDM.

[12]  Bülent Yener,et al.  Collective Sampling and Analysis of High Order Tensors for Chatroom Communications , 2006, ISI.

[13]  R. Cattell.  “Parallel proportional profiles” and other principles for determining the choice of factors by rotation , 1944 .

[14]  Hans De Sterck,et al.  A Nonlinear GMRES Optimization Algorithm for Canonical Tensor Decomposition , 2011, SIAM J. Sci. Comput..

[15]  Bülent Yener,et al.  Modeling and Multiway Analysis of Chatroom Tensors , 2005, ISI.

[16]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[17]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[18]  Yunming Ye,et al.  MultiComm: Finding Community Structure in Multi-Dimensional Networks , 2014, IEEE Transactions on Knowledge and Data Engineering.

[19]  Nikos D. Sidiropoulos,et al.  Khatri-Rao space-time codes , 2002, IEEE Trans. Signal Process..

[20]  Rasmus Bro,et al.  Recent developments in CANDECOMP/PARAFAC algorithms: a critical review , 2003 .

[21]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[22]  Andreas Zell,et al.  EDISA: extracting biclusters from multiple time-series of gene expression profiles , 2007, BMC Bioinformatics.

[23]  Brad T. Sherman,et al.  DAVID: Database for Annotation, Visualization, and Integrated Discovery , 2003, Genome Biology.

[24]  Amnon Shashua,et al.  Linear image coding for regression and classification using the tensor-rank principle , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[25]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[26]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[27]  Katsushi Ikeuchi,et al.  Appearance Based Object Modeling using Texture Database: Acquisition Compression and Rendering , 2002, Rendering Techniques.

[28]  Michael W. Berry,et al.  Discussion Tracking in Enron Email using PARAFAC. , 2008 .

[29]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .