Sequences of regressions and their independences

Ordered sequences of univariate or multivariate regressions provide statistical models for analysing data from randomized, possibly sequential interventions, from cohort or multi-wave panel studies, but also from cross-sectional or retrospective studies. Conditional independences are captured by what we name regression graphs, provided the generated distribution shares some properties with a joint Gaussian distribution. Regression graphs extend purely directed, acyclic graphs by two types of undirected graph, one type for components of joint responses and the other for components of the context vector variable. We review the special features and the history of regression graphs, prove criteria for Markov equivalence and discuss the notion of a simpler statistical covering model. Knowledge of Markov equivalence provides alternative interpretations of a given sequence of regressions, is essential for machine learning strategies and permits to use the simple graphical criteria of regression graphs on graphs for which the corresponding criteria are in general more complex. Under the known conditions that a Markov equivalent directed acyclic graph exists for any given regression graph, we give a polynomial time algorithm to find one such graph. Communicated by Domingo Morales. This invited paper is discussed in the comments available at doi:10.1007/s11749-012-0288-0, doi:10.1007/s11749-012-0287-1, doi:10.1007/s11749-012-0286-2, doi:10.1007/s11749-012-0285-3, doi:10.1007/s11749-012-0284-4. N. Wermuth ( ) Department of Mathematics, Chalmers Technical University, Gothenburg, Sweden e-mail: wermuth@chalmers.se N. Wermuth International Agency of Research on Cancer, Lyon, France K. Sadeghi Department of Statistics, University of Oxford, Oxford, UK e-mail: kayvan.sadeghi@jesus.ox.ac.uk N. Wermuth, K. Sadeghi

[1]  S. Lauritzen,et al.  Markov properties for mixed graphs , 2011, 1109.5909.

[2]  T. Richardson,et al.  Marginal log‐linear parameters for graphical Markov models , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[3]  N. Wermuth,et al.  Rejoinder on: Sequences of regressions and their independences , 2012 .

[4]  Bala Rajaratnam.  Comment on: Sequences of regressions and their independences , 2012 .

[5]  Comments on: Sequence of regressions and their independences , 2012 .

[6]  E. Lehmann,et al.  Completeness, Similar Regions, and Unbiased Estimation—Part II , 2012 .

[7]  Kayvan Sadeghi.  Markov Equivalences for Subclasses of Loopless Mixed Graphs , 2011, 1110.4539.

[8]  Generalized Hyper Markov laws for directed acyclic graphs , 2011 .

[9]  N. Wermuth,et al.  Sequences of regressions and their independences , 2011, 1110.1986.

[10]  Modelling bounded health scores with censored skew‐normal distributions , 2011, Statistics in medicine.

[11]  Kshitij Khare,et al.  Wishart distributions for decomposable covariance graph models , 2011, 1103.1768.

[12]  N. Wermuth.  PROBABILITY DISTRIBUTIONS WITH SUMMARY GRAPH STRUCTURE , 2010, 1003.3259.

[13]  M. Drton,et al.  Global identifiability of linear structural equation models , 2010, 1003.1146.

[14]  G. M. Marchetti,et al.  Chain graph models of multivariate regression type for categorical data , 2009, 0906.2098.

[15]  Tyler VanderWeele,et al.  Causality, 2nd edn , 2011 .

[16]  Flat and Multimodal Likelihoods and Model Lack of Fit in Curved Exponential Families , 2010 .

[17]  Wicher P. Bergsma,et al.  Marginal log-linear parameterization of conditional independence models , 2010 .

[18]  Josephine Yu,et al.  On a Parametrization of Positive Semidefinite Matrices with Zeros , 2010, SIAM J. Matrix Anal. Appl..

[19]  S. Sullivant,et al.  Trek separation for Gaussian graphical models , 2008, 0812.1938.

[20]  N. Wermuth,et al.  Changing Parameters by Partial Mappings , 2010 .

[21]  Jin Tian,et al.  Markov Properties for Linear Causal Models with Correlated Errors , 2009, J. Mach. Learn. Res..

[22]  Zoubin Ghahramani,et al.  The Hidden Life of Latent Variables: Bayesian Learning with Mixed Graph Models , 2009, J. Mach. Learn. Res..

[23]  P. Spirtes,et al.  MARKOV EQUIVALENCE FOR ANCESTRAL GRAPHS , 2009, 0908.3605.

[24]  N. Wermuth,et al.  Matrix representations and independencies in directed acyclic graphs , 2009, 0904.0333.

[25]  Michael Eichler,et al.  Computing Maximum Likelihood Estimates in Recursive Linear Models with Correlated Errors , 2009, J. Mach. Learn. Res..

[26]  N. Wermuth,et al.  Triangular systems for symmetric binary variables , 2009 .

[27]  Seth Sullivant,et al.  Lectures on Algebraic Statistics , 2008 .

[28]  Carlos M. Carvalho,et al.  FLEXIBLE COVARIANCE ESTIMATION IN GRAPHICAL GAUSSIAN MODELS , 2008, 0901.3267.

[29]  J. Hardt,et al.  Childhood Adversities and Suicide Attempts: A Retrospective Study , 2008, Journal of Family Violence.

[30]  A note on distortions induced by truncation with applications to linear regression systems , 2008 .

[31]  Thomas S. Richardson,et al.  Graphical Methods for Efficient Likelihood Inference in Gaussian Covariance Models , 2007, J. Mach. Learn. Res..

[32]  T. Richardson,et al.  Binary models for marginal independence , 2007, 0707.3794.

[33]  G'erard Letac,et al.  Wishart distributions for decomposable graphs , 2007, 0708.2380.

[34]  T. Richardson,et al.  Estimation of a covariance matrix with zeros , 2005, math/0508268.

[35]  Frantisek Matús,et al.  On Gaussian conditional independence structures , 2007, Kybernetika.

[36]  B. Qaqish,et al.  Multivariate logistic models , 2006 .

[37]  Milan Studený,et al.  A Graphical Representation of Equivalence Classes of AMP Chain Graphs , 2006, J. Mach. Learn. Res..

[38]  Robert Castelo,et al.  A Robust Procedure For Gaussian Graphical Model Search From Microarray Data With p Larger Than n , 2006, J. Mach. Learn. Res..

[39]  N. Wermuth,et al.  Partial inversion for linear systems and partial closure of independence graphs , 2006 .

[40]  M. Perlman,et al.  Characterizing Markov equivalence classes for AMP chain graph models , 2006, math/0607037.

[41]  N. Wermuth,et al.  Covariance chains , 2006 .

[42]  Yefim Dinitz,et al.  Dinitz' Algorithm: The Original Version and Even's Version , 2006, Essays in Memory of Shimon Even.

[43]  Jin Tian,et al.  Identifying Direct Causal Effects in Linear Models , 2005, AAAI.

[44]  N. Wermuth,et al.  On the identification of path analysis models with one hidden variable , 2005 .

[45]  Zhao Hui Zheng Zhongguo Liu Baijun,et al.  On the Markov equivalence of maximal ancestral graphs , 2005 .

[46]  M. Mouchart,et al.  Ignorable Common Information, Null sets and Basu’s First Theorem , 2005 .

[47]  M. Drton,et al.  Model selection for Gaussian concentration graphs , 2004 .

[48]  N. Wermuth,et al.  Joint response graphs and separation induced by triangular systems , 2004 .

[49]  T. Richardson,et al.  Multimodality of the likelihood in the bivariate seemingly unrelated regressions model , 2004 .

[50]  David Madigan,et al.  A graphical characterization of lattice conditional independence models , 2004, Annals of Mathematics and Artificial Intelligence.

[51]  Nanny Wermuth,et al.  A general condition for avoiding effect reversal after marginalization , 2003 .

[52]  Arno Siebes,et al.  A characterization of moral transitive acyclic directed graph Markov models as labeled trees , 2003 .

[53]  J. Pearl,et al.  A New Identification Condition for Recursive Models With Correlated Errors , 2002 .

[54]  A. Roverato.  Hyper Inverse Wishart Distribution for Non-decomposable Graphs and its Application to Bayesian Inference for Gaussian Graphical Models , 2002 .

[55]  P. Spirtes,et al.  Ancestral graph Markov models , 2002 .

[56]  D. Madigan,et al.  Alternative Markov Properties for Chain Graphs , 2001 .

[57]  Nanny Wermuth,et al.  Likelihood Factorizations for Mixed Discrete and Continuous Variables , 1999 .

[58]  Nanny Wermuth,et al.  On association models defined over independence graphs , 1998 .

[59]  G. Kauermann.  On a dualization of graphical Gaussian models , 1996 .

[60]  David Madigan,et al.  On the relation between conditional independence models determined by finite distributive lattices and by directed acyclic graphs , 1995 .

[61]  N. Wermuth,et al.  Tests of Linearity, Multivariate Normality and the Adequacy of Linear Scores , 1994 .

[62]  Judea Pearl,et al.  When can association graphs admit a causal interpretation , 1994 .

[63]  N. Wermuth,et al.  Linear Dependencies Represented by Chain Graphs , 1993 .

[64]  B. Peyton,et al.  An Introduction to Chordal Graphs and Clique Trees , 1993 .

[65]  B. Everitt.  Log-linear models for contingency tables , 1992 .

[66]  Nanny Wermuth,et al.  An approximation to maximum likelihood estimates in reduced models , 1990 .

[67]  N. Wermuth,et al.  On Substantive Research Hypotheses, Conditional Independence Graphs and Graphical Chain Models , 1990 .

[68]  Dan Geiger,et al.  Identifying independence in bayesian networks , 1990, Networks.

[69]  Judea Pearl,et al.  Equivalence and Synthesis of Causal Models , 1990, UAI.

[70]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[71]  M. Frydenberg.  The chain graph Markov property , 1990 .

[72]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[73]  S. T. Jensen,et al.  Covariance Hypotheses Which are Linear in Both the Covariance and the Inverse Covariance , 1988 .

[74]  H. Kiiveri.  An incomplete data approach to the analysis of covariance structures , 1987 .

[75]  A. Mandelbaum,et al.  Complete and Symmetrically Complete Families of Distributions , 1987 .

[76]  T. Speed,et al.  Gaussian Markov Distributions over Finite Graphs , 1986 .

[77]  Robert E. Tarjan,et al.  Simple Linear-Time Algorithms to Test Chordality of Graphs, Test Acyclicity of Hypergraphs, and Selectively Reduce Acyclic Hypergraphs , 1984, SIAM J. Comput..

[78]  T. Speed,et al.  Recursive causal models , 1984, Journal of the Australian Mathematical Society. Series A. Pure Mathematics and Statistics.

[79]  N. Wermuth,et al.  Graphical and recursive models for contingency tables , 1983 .

[80]  Catriel Beeri,et al.  On the Desirability of Acyclic Database Schemes , 1983, JACM.

[81]  N. Wermuth.  Linear Recursive Equations, Covariance Selection, and Path Analysis , 1980 .

[82]  N. Wermuth.  Model Search among Multiplicative Models , 1976 .

[83]  N. Wermuth.  Analogies between Multiplicative Models in Contingency Tables and Covariance Selection , 1976 .

[84]  T. W. Anderson.  Asymptotically Efficient Estimation of Covariance Matrices with Linear Structure , 1973 .

[85]  L. A. Goodman.  The Multivariate Analysis of Qualitative Data: Interactions among Multiple Classifications , 1970 .

[86]  J. Neyman,et al.  Research Papers in Statistics. Festschrift for J. Neyman F.N. David editor, assisted by E. Fix. London, New York, Sydney, J. Wiley & Sons, 1966, VIII p. 468 p., 105/–. , 1968, Recherches économiques de Louvain.

[87]  R. D. Bock,et al.  Analysis of covariance structures , 1966, Psychometrika.

[88]  Henri Caussinus,et al.  Contribution à l'analyse statistique des tableaux de corrélation , 1965 .

[89]  J. Darroch.  Interactions in Multi‐Factor Contingency Tables , 1962 .

[90]  T. Haavelmo.  The Statistical Implications of a System of Simultaneous Equations , 1943 .

[91]  W. G. Cochran.  The Omission or Addition of an Independent Variate in Multiple Linear Regression , 1938 .