Detecting Confounding in Multivariate Linear Models via Spectral Analysis

Abstract We study a model where one target variable Y$Y$ is correlated with a vector X:=(X1,…,Xd)$\textbf{X}:=(X_1,\dots,X_d)$ of predictor variables being potential causes of Y$Y$. We describe a method that infers to what extent the statistical dependences between X$\textbf{X}$ and Y$Y$ are due to the influence of X$\textbf{X}$ on Y$Y$ and to what extent due to a hidden common cause (confounder) of X$\textbf{X}$ and Y$Y$. The method relies on concentration of measure results for large dimensions d$d$ and an independence assumption stating that, in the absence of confounding, the vector of regression coefficients describing the influence of each X$\textbf{X}$ on Y$Y$ typically has ‘generic orientation’ relative to the eigenspaces of the covariance matrix of X$\textbf{X}$. For the special case of a scalar confounder we show that confounding typically spoils this generic orientation in a characteristic way that can be used to quantitatively estimate the amount of confounding (subject to our idealized model assumptions).

[1]  M. Bartlett.  An Inverse Matrix Adjustment Arising in Discriminant Analysis , 1951 .

[2]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[3]  B. Simon.  Trace ideals and their applications , 1979 .

[4]  S. Karlin,et al.  Classes of orderings of measures and related correlation inequalities. I. Multivariate totally positive distributions , 1980 .

[5]  G. Murphy.  C*-Algebras and Operator Theory , 1990 .

[6]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[7]  Franz von Kutschera,et al.  Causation , 1993, J. Philos. Log..

[8]  B. Simon,et al.  Rank One Perturbations with Infinitesimal Coupling , 1995 .

[9]  Christopher Meek,et al.  Strong completeness and faithfulness in Bayesian networks , 1995, UAI.

[10]  Barry Simon,et al.  Spectral analysis of rank one perturbations and applications , 1995 .

[11]  E. Ecer,et al.  Numerical Linear Algebra and Applications , 1995, IEEE Computational Science and Engineering.

[12]  M. Rudelson.  Random Vectors in the Isotropic Position , 1996, math/9608208.

[13]  S. Albeverio,et al.  Rank One Perturbations, Approximations, and Selfadjoint Extensions , 1997 .

[14]  Lotharingien de Combinatoire.  Free Probability Theory and Non-crossing Partitions , 1997 .

[15]  I-Cheng Yeh,et al.  Modeling of strength of high-performance concrete using artificial neural networks , 1998 .

[16]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[17]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[18]  S. Albeverio,et al.  Rank one perturbations , 2000 .

[19]  The Aronszajn–Donoghue Theory for Rank One Perturbations of the $$\mathcal{H}_{-2} {\text{-Class}}$$ , 2004 .

[20]  Bernhard Schölkopf,et al.  Kernel Methods for Measuring Independence , 2005, J. Mach. Learn. Res..

[21]  J. Cima,et al.  The Cauchy Transform , 2006 .

[22]  The Cauchy Transform (Mathematical Surveys and Monographs 125): BOOK REVIEWS , 2007 .

[23]  Patrik O. Hoyer,et al.  Estimation of causal effects using linear non-Gaussian causal models with hidden variables , 2008, Int. J. Approx. Reason..

[24]  Dominik Janzing,et al.  Justifying Additive Noise Model-Based Causal Discovery via Algorithmic Information Theory , 2009, Open Syst. Inf. Dyn..

[25]  Bernhard Schölkopf,et al.  Telling cause from effect based on high-dimensional observations , 2009, ICML.

[26]  Pablo A. Parrilo,et al.  Latent variable graphical model selection via convex optimization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[27]  R. Vershynin.  How Close is the Sample Covariance Matrix to the Actual Covariance Matrix? , 2010, 1004.3484.

[28]  Bernhard Schölkopf,et al.  Causal Inference Using the Algorithmic Markov Condition , 2008, IEEE Transactions on Information Theory.

[29]  Dominik Janzing,et al.  Testing whether linear equations are causal: A free probability theory approach , 2011, UAI.

[30]  Michael P. Murray,et al.  Instrumental Variables , 2011, International Encyclopedia of Statistical Science.

[31]  Bernhard Schölkopf,et al.  Detecting low-complexity unobserved causes , 2011, UAI.

[32]  A. Guionnet,et al.  Free probability and random matrices , 2012 .

[33]  Jan Lemeire,et al.  Replacing Causal Faithfulness with Algorithmic Independence of Conditionals , 2013, Minds and Machines.

[34]  Moritz Grosse-Wentrup,et al.  Quantifying causal influences , 2012, 1203.6502.

[35]  Peter Buhlmann,et al.  Geometry of the faithfulness assumption in causal inference , 2012, 1207.0547.

[36]  Shaun M. Fallat,et al.  Total positivity in Markov structures , 2015, 1510.01290.

[37]  Marvin A. Carlson.  Editor , 2015 .

[38]  Claudia Baier.  Direction Of Time , 2016 .

[39]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .