Estimating and Controlling the False Discovery Rate of the PC Algorithm Using Edge-specific P-Values

Many causal discovery algorithms infer graphical structure from observational data. The PC algorithm in particular estimates a completed partially directed acyclic graph (CPDAG), or an acyclic graph containing directed edges identifiable with conditional independence testing. However, few groups have investigated strategies for estimating and controlling the false discovery rate (FDR) of the edges in the CPDAG. In this article, we introduce PC with p-values (PC-p), a fast algorithm that robustly computes edge-specific p-values and then estimates and controls the FDR across the edges. PC-p specifically uses the p-values returned by many conditional independence (CI) tests to upper bound the p-values of more complex edge-specific hypothesis tests. The algorithm then estimates and controls the FDR using the bounded p-values and the Benjamini-Yekutieli FDR procedure. Modifications to the original PC algorithm also help PC-p accurately compute the upper bounds despite non-zero Type II error rates. Experiments show that PC-p yields more accurate FDR estimation and control across the edges in a variety of CPDAGs compared to alternative methods.

[1]  Kush R. Varshney,et al.  Structure Learning from Time Series with False Discovery Control , 2018, ArXiv.

[2]  J Runge,et al.  Causal network reconstruction from time series: From theoretical assumptions to practical estimation. , 2018, Chaos.

[3]  Peter Bühlmann,et al.  Estimating High-Dimensional Directed Acyclic Graphs with the PC-Algorithm , 2007, J. Mach. Learn. Res..

[4]  Thomas S. Richardson,et al.  Causal Inference in the Presence of Latent Variables and Selection Bias , 1995, UAI.

[5]  Ioannis Tsamardinos,et al.  A unified approach to estimation and control of the False Discovery Rate in Bayesian network skeleton identification , 2011, ESANN.

[6]  Gregory F. Cooper,et al.  Causal Discovery from a Mixture of Experimental and Observational Data , 1999, UAI.

[7]  Ioannis Tsamardinos,et al.  Estimation and Control of the False Discovery Rate of Bayesian Network Skeleton Identification , 2014 .

[8]  Wei Sun,et al.  PenPC: A two‐step approach to estimate the skeletons of high‐dimensional directed acyclic graphs , 2014, Biometrics.

[9]  Jiuyong Li,et al.  Inferring microRNA-mRNA causal regulatory relationships from expression data , 2013, Bioinform..

[10]  Nir Friedman,et al.  Data Analysis with Bayesian Networks: A Bootstrap Approach , 1999, UAI.

[11]  Laura E. Brown,et al.  Bounding the False Discovery Rate in Local Bayesian Network Learning , 2008, AAAI.

[12]  Christopher Meek,et al.  Strong completeness and faithfulness in Bayesian networks , 1995, UAI.

[13]  Qing Zhou,et al.  Concave penalized estimation of sparse Gaussian Bayesian networks , 2014, J. Mach. Learn. Res..

[14]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[15]  P. Spirtes,et al.  Causation, Prediction, and Search, 2nd Edition , 2001 .

[16]  Martin Wainwright,et al.  Search for Causal Models , 2018 .

[17]  Qing Zhou,et al.  Learning Large-Scale Bayesian Networks with the sparsebn Package , 2017, Journal of Statistical Software.

[18]  Reiji Teramoto,et al.  Estimating causal effects with a non-paranormal method for the design of efficient intervention experiments , 2014, BMC Bioinformatics.

[19]  Xintao Hu,et al.  Inferring consistent functional interaction patterns from natural stimulus FMRI data , 2012, NeuroImage.

[20]  M. Fréchet.  Généralisation du théorème des probabilités totales , 1935 .

[21]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[22]  Diego Colombo,et al.  Order-independent constraint-based causal structure learning , 2012, J. Mach. Learn. Res..

[23]  Martin J. McKeown,et al.  Learning brain connectivity with the false-discovery-rate-controlled PC-algorithm , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[24]  Pradeep Ravikumar,et al.  DAGs with NO TEARS: Continuous Optimization for Structure Learning , 2018, NeurIPS.

[25]  Kim-Anh Do,et al.  Prognostic Gene Signature Identification Using Causal Structure Learning: Applications in Kidney Cancer , 2015, Cancer informatics.

[26]  Ruocheng Guo,et al.  A Survey of Learning Causality with Data , 2018, ACM Comput. Surv..

[27]  K. Sachs,et al.  Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data , 2005, Science.

[28]  Naftali Harris,et al.  PC algorithm for nonparanormal graphical models , 2013, J. Mach. Learn. Res..

[29]  Daniel Müllensiefen,et al.  Investigating the importance of self-theories of intelligence and musicality for students' academic and musical achievement , 2015, Front. Psychol..

[30]  Sergey M. Plis,et al.  Learning Dynamic Structure from Undersampled Data , 2017 .

[31]  Kathleen M. Gates,et al.  Inferring functional connectivity in MRI using Bayesian network structure learning with a modified PC algorithm , 2013, NeuroImage.

[32]  Aapo Hyvärinen,et al.  DirectLiNGAM: A Direct Method for Learning a Linear Non-Gaussian Structural Equation Model , 2011, J. Mach. Learn. Res..

[33]  M. Zey,et al.  On corporate structure, strategy, and performance: a study with directed acyclic graphs and PC algorithm , 2010 .

[34]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[35]  Bernhard Schölkopf,et al.  Nonlinear causal discovery with additive noise models , 2008, NIPS.

[36]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[37]  Thomas S. Richardson,et al.  A Discovery Algorithm for Directed Cyclic Graphs , 1996, UAI.

[38]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[39]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[40]  Z. Jane Wang,et al.  Controlling the False Discovery Rate of the Association/Causality Structure Learned with the PC Algorithm , 2009, J. Mach. Learn. Res..

[41]  Xintao Wu,et al.  Exploring gene causal interactions using an enhanced constraint-based method , 2006, Pattern Recognit..

[42]  Arthur W. Toga,et al.  Bayesian approach for network modeling of brain structural features , 2010, Medical Imaging.

[43]  Jüri Lember,et al.  Bridging Viterbi and posterior decoding: a generalized risk approach to hidden path inference based on hidden Markov models , 2014, J. Mach. Learn. Res..

[44]  David Heckerman,et al.  Determining the Number of Non-Spurious Arcs in a Learned DAG Model: Investigation of a Bayesian and a Frequentist Approach , 2007, UAI.

[45]  Aapo Hyvärinen,et al.  A Linear Non-Gaussian Acyclic Model for Causal Discovery , 2006, J. Mach. Learn. Res..