CANDECOMP/PARAFAC model order selection based on Reconstruction Error in the presence of Kronecker structured colored noise

Canonical Decomposition (CANDECOMP) also known as Parallel Factor Analysis (PARAFAC) is a well-known multiway model in high-dimensional data modeling. Approaches that use CANDECOMP/PARAFAC for parametric modeling of a noisy observation require an estimate of the number of signal components (rank) of the data as well. In real applications, the true model of data is unknown and model order selection is a challenging step of these algorithms. In addition, considering noise samples with correlation in different dimensions makes the model order selection even more challenging. Model order selection methods generally minimize a criterion to find the optimum model order. In this paper, we propose using the Reconstruction error, which is the error between the reconstructed data and the unavailable noiseless data, for a range of possible ranks, and use an estimate of this error as the desired criterion for order selection. Furthermore, we propose using the CORCONDIA measure for determining the range of possible model orders. In the presence of the colored noise with Kronecker structure, our proposed algorithm performs the multidimensional prewhitening prior to the model order selection. In addition, our method is able to estimate the noise covariance through an iterative algorithm when no prior information about the noise covariance is available. Simulation results show that the proposed method can be effectively exploited for robustly detecting the true rank of the observed tensor even in mid and low SNRs (i.e. 0-10 dB). It also has an advantage over the state-of-the-art methods, such as different variants of CORCONDIA, by having a better Probability of Detection (PoD) with almost no extra computational overhead after the CANDECOMP/PARAFAC decomposition.

[1]  Bülent Yener,et al.  Collective Sampling and Analysis of High Order Tensors for Chatroom Communications , 2006, ISI.

[2]  Munther A. Dahleh,et al.  Noisy Data and Impulse Response Estimation , 2010, IEEE Transactions on Signal Processing.

[3]  Fumikazu Miwakeichi,et al.  Decomposing EEG data into space–time–frequency components using Parallel Factor Analysis , 2004, NeuroImage.

[4]  R. Bro,et al.  Practical aspects of PARAFAC modeling of fluorescence excitation‐emission data , 2003 .

[5]  Lourens J. Waldorp,et al.  Spatiotemporal EEG/MEG source analysis based on a parametric noise covariance model , 2002, IEEE Transactions on Biomedical Engineering.

[6]  Rasmus Bro,et al.  Recent developments in CANDECOMP/PARAFAC algorithms: a critical review , 2003 .

[7]  M. Haardt,et al.  Robust methods based on the hosvd for estimating the model order in PARAFAC models , 2008, 2008 5th IEEE Sensor Array and Multichannel Signal Processing Workshop.

[8]  Florian Roemer,et al.  Efficient source enumeration for accurate direction-of-arrival estimation in threshold region , 2013, Digit. Signal Process..

[9]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[10]  Martin Haardt,et al.  Multidimensional prewhitening for enhanced signal reconstruction and parameter estimation in colored noise with Kronecker correlation structure , 2013, Signal Process..

[11]  Franz J. Király,et al.  Approximate rank-detecting factorization of low-rank tensors , 2012, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Lei Huang,et al.  Core consistency diagnostic aided by reconstruction error for accurate enumeration of the number of components in parafac models , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Andrzej Cichocki,et al.  CANDECOMP/PARAFAC Decomposition of High-Order Tensors Through Tensor Reshaping , 2012, IEEE Transactions on Signal Processing.

[14]  Sos S. Agaian,et al.  Multiresolution Decomposition Schemes Using the Parameterized Logarithmic Image Processing Model with Application to Image Fusion , 2011, EURASIP J. Adv. Signal Process..

[15]  Carlos H. Muravchik,et al.  Shrinkage Approach for Spatiotemporal EEG Covariance Matrix Estimation , 2013, IEEE Transactions on Signal Processing.

[16]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[17]  H. Kiers,et al.  Three-mode principal components analysis: choosing the numbers of components and sensitivity to local optima. , 2000, The British journal of mathematical and statistical psychology.

[18]  Zbynek Koldovský,et al.  Cramér-Rao-Induced Bounds for CANDECOMP/PARAFAC Tensor Decomposition , 2012, IEEE Transactions on Signal Processing.

[19]  Florian Roemer,et al.  Multi-dimensional model order selection , 2011, EURASIP J. Adv. Signal Process..

[20]  Hongzhe Li,et al.  Model selection and estimation in the matrix normal graphical model , 2012, J. Multivar. Anal..

[21]  Josef A. Nossek,et al.  Estimation of rank deficient covariance matrices with Kronecker structure , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Florian Roemer,et al.  Enhanced Model Order Estimation in Colored Noise Scenarios via Noise Prewhitening , 2011 .

[23]  André Quinquis,et al.  A New Method for Estimating the Number of Harmonic Components in Noise with Application in High Resolution Radar , 2004, EURASIP J. Adv. Signal Process..

[24]  Bülent Yener,et al.  Unsupervised Multiway Data Analysis: A Literature Survey , 2009, IEEE Transactions on Knowledge and Data Engineering.

[25]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[26]  Gérard Favier,et al.  An Algebraic Solution for the Candecomp/PARAFAC Decomposition with Circulant Factors , 2014, SIAM J. Matrix Anal. Appl..

[27]  L. K. Hansen,et al.  Automatic relevance determination for multi‐way models , 2009 .

[28]  R. Bro,et al.  A new efficient method for determining the number of components in PARAFAC models , 2003 .

[29]  Lars Kai Hansen,et al.  Parallel Factor Analysis as an exploratory tool for wavelet transformed event-related EEG , 2006, NeuroImage.

[30]  H. Kiers Towards a standardized notation and terminology in multiway analysis , 2000 .

[31]  Lieven De Lathauwer,et al.  Tensor-based techniques for the blind separation of DS-CDMA signals , 2007, Signal Process..

[32]  Andrzej Cichocki,et al.  Canonical Polyadic Decomposition Based on a Single Mode Blind Source Separation , 2012, IEEE Signal Processing Letters.

[33]  Lei Huang,et al.  A multi-dimensional model order selection criterion with improved identifiability , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Florian Roemer,et al.  DETERMINISTIC PREWHITENING TO IMPROVE SUBSPACE BASED PARAMETER ESTIMATION TECHNIQUES IN SEVERELY COLORED NOISE ENVIRONMENTS , 2009 .

[35]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Laurent Albera,et al.  Iterative methods for the canonical decomposition of multi-way arrays: Application to blind underdetermined mixture identification , 2011, Signal Process..

[37]  Michael W. Berry,et al.  Discussion Tracking in Enron Email using PARAFAC. , 2008 .

[38]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[39]  Andrzej Cichocki,et al.  Deflation method for CANDECOMP/PARAFAC tensor decomposition , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Alfred O. Hero,et al.  Kronecker sum decompositions of space-time data , 2013, 2013 5th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[41]  Petre Stoica,et al.  On Estimation of Covariance Matrices With Kronecker Product Structure , 2008, IEEE Transactions on Signal Processing.

[42]  H. Kiers,et al.  Selecting among three-mode principal component models of different types and complexities: a numerical convex hull based method. , 2006, The British journal of mathematical and statistical psychology.

[43]  Magnus Jansson,et al.  On Toeplitz and Kronecker structured covariance matrix estimation , 2010, 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop.

[44]  Fetsje Bijma,et al.  The spatiotemporal MEG covariance matrix modeled as a sum of Kronecker products , 2005, NeuroImage.

[45]  Kyuwan Choi,et al.  Detecting the Number of Clusters in n-Way Probabilistic Clustering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Alfred O. Hero,et al.  Covariance Estimation in High Dimensions Via Kronecker Product Expansions , 2013, IEEE Transactions on Signal Processing.

[47]  William S Rayens,et al.  Structure-seeking multilinear methods for the analysis of fMRI data , 2004, NeuroImage.

[48]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..