Dynamic Multivariate Functional Data Modeling via Sparse Subspace Learning

ABSTRACT Multivariate functional data from a complex system are naturally high-dimensional and have a complex cross-correlation structure. The complexity of data structure can be observed as that (1) some functions are strongly correlated with similar features, while some others may have almost no cross-correlations with quite diverse features; and (2) the cross-correlation structure may also change over time due to the system evolution. With this regard, this article presents a dynamic subspace learning method for multivariate functional data modeling. In particular, we consider that different functions come from different subspaces, and only functions of the same subspace have cross-correlations with each other. The subspaces can be automatically formulated and learned by reformatting the problem as a sparse regression. By allowing but regularizing the regression change over time, we can describe the cross-correlation dynamics. The model can be efficiently estimated by the fast iterative shrinkage-thresholding algorithm, and the features of each subspace can be extracted using the smooth multi-channel functional principal component analysis. Some theoretical properties of the model are presented. Numerical studies, together with case studies, demonstrate the efficiency and applicability of the proposed methodology.

[1]  Jieping Ye,et al.  An efficient algorithm for a class of fused lasso problems , 2010, KDD.

[2]  T. Hsing,et al.  Canonical correlation for stochastic processes , 2008 .

[3]  Xinghao Qiao,et al.  Functional Graphical Models , 2018, Journal of the American Statistical Association.

[4]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[5]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[6]  B. Caffo,et al.  MULTILEVEL FUNCTIONAL PRINCIPAL COMPONENT ANALYSIS. , 2009, The annals of applied statistics.

[7]  Jianhua Z. Huang,et al.  Functional principal components analysis via penalized rank one approximation , 2008, 0807.4862.

[8]  E. Hannan The general theory of canonical correlation and its relation to functional analysis , 1961, Journal of the Australian Mathematical Society.

[9]  Antonio Cuevas,et al.  On Mahalanobis Distance in Functional Settings , 2018, J. Mach. Learn. Res..

[10]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[11]  Peihua Qiu,et al.  A Change-Point Approach for Phase-I Analysis in Multivariate Profile Monitoring and Diagnosis , 2016, Technometrics.

[12]  LiaoZicheng,et al.  Discriminative hierarchical part-based models for human parsing and action recognition , 2012 .

[13]  H. Müller,et al.  Dynamical Correlation for Multivariate Longitudinal Data , 2005 .

[14]  Eric P. Xing,et al.  On Time Varying Undirected Graphs , 2011, AISTATS.

[15]  R. Tibshirani,et al.  A fused lasso latent feature model for analyzing multi-sample aCGH data. , 2011, Biostatistics.

[16]  Han Liu,et al.  Joint estimation of multiple graphical models from high dimensional time series , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[17]  Mladen Kolar,et al.  Estimating networks with jumps. , 2010, Electronic journal of statistics.

[18]  Wen Gao,et al.  Efficient Generalized Fused Lasso and Its Applications , 2016, ACM Trans. Intell. Syst. Technol..

[19]  Larry A. Wasserman,et al.  Time varying undirected graphs , 2008, Machine Learning.

[20]  Arkadi Nemirovski,et al.  EFFICIENT METHODS IN CONVEX PROGRAMMING , 2007 .

[21]  Dongdong Xiang,et al.  NONPARAMETRIC REGRESSION ANALYSIS OF MULTIVARIATE LONGITUDINAL DATA , 2013 .

[22]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[23]  Geert Verbeke,et al.  Pairwise Fitting of Mixed Models for the Joint Modeling of Multivariate Longitudinal Profiles , 2006, Biometrics.

[24]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[25]  Jeng-Min Chiou,et al.  A pairwise interaction model for multivariate functional and longitudinal data. , 2016, Biometrika.

[26]  Ulrich Stadtmüller,et al.  Functional singular component analysis , 2011 .

[27]  David B. Dunson,et al.  Bayesian Graphical Models for Multivariate Functional Data , 2014, J. Mach. Learn. Res..

[28]  Huan Xu,et al.  Noisy Sparse Subspace Clustering , 2013, J. Mach. Learn. Res..

[29]  Jeng-Min Chiou,et al.  Linear manifold modelling of multivariate functional data , 2014 .

[30]  I. Gohberg,et al.  Basic Operator Theory , 1981 .

[31]  Xinghao Qiao,et al.  Doubly functional graphical models in high dimensions , 2020, Biometrika.

[32]  Yui Man Lui,et al.  Human gesture recognition on product manifolds , 2012, J. Mach. Learn. Res..

[33]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[34]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[35]  Ran Gilad-Bachrach,et al.  Full body gait analysis with Kinect , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[36]  Hao Yan,et al.  Weakly correlated profile monitoring based on sparse multi-channel functional principal component analysis , 2018, IISE Transactions.

[37]  Hao Yan,et al.  Multiple profiles sensor-based monitoring and anomaly detection , 2018, Journal of Quality Technology.

[38]  Jeng-Min Chiou,et al.  Multivariate functional principal component analysis: A normalization approach , 2014 .

[39]  Yan Liu,et al.  Functional Subspace Clustering with Application to Time Series , 2015, ICML.

[40]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[41]  Q. Yao,et al.  Modelling multiple time series via common factors , 2008 .

[42]  Z. Harchaoui,et al.  Multiple Change-Point Estimation With a Total Variation Penalty , 2010 .

[43]  B. Li,et al.  A Nonparametric Graphical Model for Functional Data With Application to Brain Networks Based on fMRI , 2018, Journal of the American Statistical Association.

[44]  B. Silverman,et al.  Canonical correlation analysis when the data are curves. , 1993 .