Non-Independent Components Analysis

A seminal result in the ICA literature states that for $AY = \varepsilon$, if the components of $\varepsilon$ are independent and at most one is Gaussian, then $A$ is identified up to sign and permutation of its rows (Comon, 1994). In this paper we study to which extent the independence assumption can be relaxed by replacing it with restrictions on higher order moment or cumulant tensors of $\varepsilon$. We document new conditions that establish identification for several non-independent component models, e.g. common variance models, and propose efficient estimation methods based on the identification results. We show that in situations where independence cannot be assumed the efficiency gains can be significant relative to methods that rely on independence.

[1]  G. Bekaert,et al.  Identifying Aggregate Demand and Supply Shocks Using Sign Restrictions and Higher-Order Moments , 2021 .

[2]  Emanuele Taufer,et al.  Asymptotic theory for statistics based on cumulant vectors with applications , 2021, Scandinavian Journal of Statistics.

[3]  G. Bekaert,et al.  Macro Risks and the Term Structure of Interest Rates , 2016, Journal of Financial Economics.

[4]  Piotr Zwiernik,et al.  Semialgebraic Statistics and Latent Tree Models , 2015 .

[5]  David S. Matteson,et al.  Independent Component Analysis via Distance Covariance , 2013, 1306.4911.

[6]  M. Yuan,et al.  Independent component analysis via nonparametric maximum likelihood estimation , 2012, 1206.0457.

[7]  Klaus Nordhausen,et al.  A New Performance Index for ICA: Properties, Computation and Asymptotic Analysis , 2010, LVA/ICA.

[8]  G. Guarino,et al.  A new method for fast computing unbiased estimators of cumulants , 2008, Stat. Comput..

[9]  M. Genton,et al.  Robust Likelihood Methods Based on the Skew‐t and Related Distributions , 2008 .

[10]  Maria L. Rizzo,et al.  Measuring and testing dependence by correlation of distances , 2007, 0803.4101.

[11]  David A. Cox,et al.  Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3/e (Undergraduate Texts in Mathematics) , 2007 .

[12]  P. Bickel,et al.  Efficient independent component analysis , 2006, 0705.4230.

[13]  Aapo Hyvärinen,et al.  A Linear Non-Gaussian Acyclic Model for Causal Discovery , 2006, J. Mach. Learn. Res..

[14]  M. McAleer,et al.  Multivariate Stochastic Volatility: A Review , 2006 .

[15]  Michael I. Jordan,et al.  Beyond Independent Components: Trees and Clusters , 2003, J. Mach. Learn. Res..

[16]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[17]  Aapo Hyvärinen,et al.  Topographic Independent Component Analysis , 2001, Neural Computation.

[18]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[19]  Andrew D. Back,et al.  A First Application of Independent Component Analysis to Extracting Structure from Stock Returns , 1997, Int. J. Neural Syst..

[20]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[21]  Jean-Francois Cardoso,et al.  Source separation using higher order moments , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[22]  T. Speed Cumulants and partition lattices II: generalised k-statistics , 1986, Journal of the Australian Mathematical Society. Series A. Pure Mathematics and Statistics.

[23]  Terence P. Speed,et al.  CUMULANTS AND PARTITION LATTICES1 , 1983 .

[24]  L. Hansen Large Sample Properties of Generalized Method of Moments Estimators , 1982 .

[25]  David R. Brillinger,et al.  The calculation of cumulants via conditioning , 1969 .

[26]  Lynne Seymour,et al.  Institute of Mathematical Statistics LECTURE NOTES ? MONOGRAPH SERIES , 2016 .

[27]  Robert Tibshirani,et al.  Independent Components Analysis through Product Density Estimation , 2002, NIPS.

[28]  Jean-Franois Cardoso High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[29]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[30]  C. Radhakrishna Rao,et al.  Linear Statistical Inference and its Applications, Second Editon , 1973, Wiley Series in Probability and Statistics.

[31]  Rory A. Fisher,et al.  Moments and Product Moments of Sampling Distributions , 1930 .

[32]  Xiaofeng Shao,et al.  Martingale Difference Correlation and Its Use in High-Dimensional Variable Screening , 2014 .