PARAFAC2 AO-ADMM: Constraints in all modes

The PARAFAC2 model provides a flexible alternative to the popular CANDECOMP/PARAFAC (CP) model for tensor decompositions. Unlike CP, PARAFAC2 allows factor matrices in one mode (i.e., evolving mode) to change across tensor slices, which has proven useful for applications in different domains such as chemometrics, and neuroscience. However, the evolving mode of the PARAFAC2 model is traditionally modelled implicitly, which makes it challenging to regularise it. Currently, the only way to apply regularisation on that mode is with a flexible coupling approach, which finds the solution through regularised least squares subproblems. In this work, we instead propose an alternating direction method of multipliers (ADMM)-based algorithm for fitting PARAFAC2 and widen the possible regularisation penalties to any proximable function. Our experiments demonstrate that the proposed ADMM-based approach for PARAFAC2 can accurately recover the underlying components from simulated data while being both computationally efficient and flexible in terms of imposing constraints.

[1]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[2]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[3]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[4]  R. Bro PARAFAC. Tutorial and applications , 1997 .

[5]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[6]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[7]  Jimeng Sun,et al.  COPA: Constrained PARAFAC2 for Sparse & Large Datasets , 2018, CIKM.

[8]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[9]  Jeremy E. Cohen,et al.  An Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorization , 2021, 2020 28th European Signal Processing Conference (EUSIPCO).

[10]  R. Bro,et al.  PARAFAC2—Part I. A direct fitting algorithm for the PARAFAC2 model , 1999 .

[11]  Tamara G. Kolda,et al.  Cross-language information retrieval using PARAFAC2 , 2007, KDD '07.

[12]  Nikos D. Sidiropoulos,et al.  A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization , 2015, IEEE Transactions on Signal Processing.

[13]  Jimeng Sun,et al.  LogPar: Logistic PARAFAC2 Factorization for Temporal Binary Data with Missing Values , 2020, KDD.

[14]  Claus A. Andersson,et al.  PARAFAC2—Part II. Modeling chromatographic data with retention time shifts , 1999 .

[15]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[16]  Rasmus Bro,et al.  Nonnegative PARAFAC2: A Flexible Coupling Approach , 2018, LVA/ICA.

[17]  Tulay Adali,et al.  Tracing Network Evolution Using The Parafac2 Model , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Nathaniel E. Helwig,et al.  Estimating latent trends in multivariate longitudinal data via Parafac2 with functional and structural constraints , 2017, Biometrical journal. Biometrische Zeitschrift.

[19]  Laurent Condat,et al.  A Direct Algorithm for 1-D Total Variation Denoising , 2013, IEEE Signal Processing Letters.

[20]  Rasmus Bro,et al.  Multiway analysis of epilepsy tensors , 2007, ISMB/ECCB.

[21]  Nathaniel E. Helwig,et al.  The Special Sign Indeterminacy of the Direct-Fitting Parafac2 Model: Some Implications, Cautions, and Recommendations for Simultaneous Component Analysis , 2013, Psychometrika.

[22]  Lars Kai Hansen,et al.  Parallel Factor Analysis as an exploratory tool for wavelet transformed event-related EEG , 2006, NeuroImage.