Stability in GRN Inference.

Reconstructing a gene regulatory network from one or more sets of omics measurements has been a major task of computational biology in the last 20 years. Despite an overwhelming number of algorithms proposed to solve the network inference problem either in the general scenario or in an ad-hoc tailored situation, assessing the stability of reconstruction is still an uncharted territory and exploratory studies mainly tackled theoretical aspects. We introduce here empirical stability, which is induced by variability of reconstruction as a function of data subsampling. By evaluating differences between networks that are inferred using different subsets of the same data we obtain quantitative indicators of the robustness of the algorithm, of the noise level affecting the data, and, overall, of the reliability of the reconstructed graph. We show that empirical stability can be used whenever no ground truth is available to compute a direct measure of the similarity between the inferred structure and the true network. The main ingredient here is a suite of indicators, called NetSI, providing statistics of distances between graphs generated by a given algorithm fed with different data subsets, where the chosen metric is the Hamming-Ipsen-Mikhailov (HIM) distance evaluating dissimilarity of graph topologies with shared nodes. Operatively, the NetSI family is demonstrated here on synthetic and high-throughput datasets, inferring graphs at different resolution levels (topology, direction, weight), showing how the stability indicators can be effectively used for the quantitative comparison of the stability of different reconstruction algorithms.

[1]  Rudiyanto Gunawan,et al.  SINCERITIES: inferring gene regulatory networks from time-stamped single cell transcriptional expression profiles , 2016, bioRxiv.

[2]  Abhijeet Pataskar,et al.  Computational challenges in modeling gene regulatory events , 2016, Transcription.

[3]  Fei Liu,et al.  Inference of Gene Regulatory Network Based on Local Bayesian Networks , 2016, PLoS Comput. Biol..

[4]  Chris Wiggins,et al.  ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context , 2004, BMC Bioinformatics.

[5]  Jean-Philippe Vert,et al.  TIGRESS: Trustful Inference of Gene REgulation using Stability Selection , 2012, BMC Systems Biology.

[6]  Manuel Sanchez-Castillo,et al.  A Bayesian framework for the inference of gene regulatory networks from time and pseudo‐time series data , 2018, Bioinform..

[7]  P. Geurts,et al.  Inferring Regulatory Networks from Expression Data Using Tree-Based Methods , 2010, PloS one.

[8]  Cesare Furlanello,et al.  Sparse Predictive Structure of Deconvolved Functional Brain Networks , 2013, 1310.6547.

[9]  Philippe Salembier,et al.  NetBenchmark: a bioconductor package for reproducible benchmarks of gene regulatory network inference , 2015, BMC Bioinformatics.

[10]  J. Collins,et al.  Large-Scale Mapping and Validation of Escherichia coli Transcriptional Regulation from a Compendium of Expression Profiles , 2007, PLoS biology.

[11]  Francesc Sagués,et al.  Robustness and Stability of the Gene Regulatory Network Involved in DV Boundary Formation in the Drosophila Wing , 2007, PloS one.

[12]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[13]  Yan Zhang,et al.  Inference of time-delayed gene regulatory networks based on dynamic Bayesian network hybrid learning method , 2017, Oncotarget.

[14]  A. Guiseppi-Elie,et al.  Stable Gene Regulatory Network Modeling From Steady-State Data † , 2016, Bioengineering.

[15]  Hazem N. Nounou,et al.  An Overview of the Statistical Methods Used for Inferring Gene Regulatory Networks and Protein-Protein Interaction Networks , 2013, Adv. Bioinformatics.

[16]  Xiao-Jiang Feng,et al.  Identifying Biological Network Structure, Predicting Network Behavior, and Classifying Network State With High Dimensional Model Representation (HDMR) , 2012, PloS one.

[17]  Roberto Visintainer,et al.  Distances and Stability in Biological Network Theory , 2013 .

[18]  Carlos H. A. Higa,et al.  Inference of Gene Regulatory Networks Using Coefficient of Determination, Tsallis Entropy and Biological Prior Knowledge , 2016, 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering (BIBE).

[19]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[20]  K. Basso,et al.  A systems biology approach to prediction of oncogenes and molecular perturbation targets in B-cell lymphomas , 2008, Molecular systems biology.

[21]  Matthias Dehmer,et al.  B-cell lymphoma gene regulatory networks: biological consistency among inference methods , 2013, Front. Genet..

[22]  Nicole Radde,et al.  Inferring Gene Regulatory Networks from Expression Data , 2019 .

[23]  Frank Emmert-Streib,et al.  Revealing differences in gene network inference algorithms on the network level by ensemble methods , 2010, Bioinform..

[24]  Steve Horvath,et al.  WGCNA: an R package for weighted correlation network analysis , 2008, BMC Bioinformatics.

[25]  Michael Hecker,et al.  Gene regulatory network inference: Data integration in dynamic models - A review , 2009, Biosyst..

[26]  Paul Pavlidis,et al.  The role of indirect connections in gene networks in predicting function , 2011, Bioinform..

[27]  Sandra Heise,et al.  TRaCE+: Ensemble inference of gene regulatory networks from transcriptional expression profiles of gene knock-out experiments , 2016, BMC Bioinformatics.

[28]  S. Horvath.  Weighted Network Analysis: Applications in Genomics and Systems Biology , 2011 .

[29]  G. Altay,et al.  Empirically determining the sample size for large-scale gene network inference algorithms. , 2012, IET systems biology.

[30]  Ruth Nussinov,et al.  Structure and dynamics of molecular networks: A novel paradigm of drug discovery. A comprehensive review , 2012, Pharmacology & therapeutics.

[31]  Abdollah Homaifar,et al.  Inferring stable gene regulatory networks from steady-state data , 2015, 2015 41st Annual Northeast Biomedical Engineering Conference (NEBEC).

[32]  William Chad Young,et al.  Integration of multiple data sources for gene network inference using genetic perturbation data , 2017, bioRxiv.

[33]  Qinke Peng,et al.  Inference of Gene Regulatory Networks Using Bayesian Nonparametric Regression and Topology Information , 2017, Comput. Math. Methods Medicine.

[34]  K. Basso,et al.  BCL6 interacts with the transcription factor Miz-1 to suppress the cyclin-dependent kinase inhibitor p21 and cell cycle arrest in germinal center B cells , 2005, Nature Immunology.

[35]  Cesare Furlanello,et al.  Algebraic Comparison of Partial Lists in Bioinformatics , 2010, PloS one.

[36]  Edward R. Dougherty,et al.  Validation of gene regulatory networks: scientific and inferential , 2011, Briefings Bioinform..

[37]  S. Shen-Orr,et al.  Network motifs in the transcriptional regulation network of Escherichia coli , 2002, Nature Genetics.

[38]  Cesare Furlanello,et al.  An introduction to spectral distances in networks , 2010, WIRN.

[40]  Julio R. Banga,et al.  Inference of complex biological networks: distinguishability issues and optimization-based solutions , 2011, BMC Systems Biology.

[41]  Steve Horvath,et al.  Weighted Network Analysis , 2011 .

[42]  Adam A. Margolin,et al.  Reverse engineering of regulatory networks in human B cells , 2005, Nature Genetics.

[43]  R. A. Brooker.  The Autocode Programs developed for the Manchester University Computers , 1958, Comput. J..

[44]  D. Floreano,et al.  Revealing strengths and weaknesses of methods for gene network inference , 2010, Proceedings of the National Academy of Sciences.

[45]  Thalia E. Chan,et al.  Gene Regulatory Network Inference from Single-Cell Data Using Multivariate Information Measures , 2016, bioRxiv.

[46]  Albert-László Barabási,et al.  Controllability of complex networks , 2011, Nature.

[47]  Alberto Franzin,et al.  bnstruct: an R package for Bayesian Network structure learning in the presence of missing data , 2016, Bioinform..

[48]  Wei Liu,et al.  Improving gene regulatory network structure using redundancy reduction in the MRNET algorithm , 2017 .

[49]  Riet De Smet,et al.  Advantages and limitations of current network inference methods , 2010, Nature Reviews Microbiology.

[50]  Michael Banf,et al.  Computational inference of gene regulatory networks: Approaches, limitations and opportunities. , 2017, Biochimica et biophysica acta. Gene regulatory mechanisms.

[51]  Frank Emmert-Streib,et al.  Bagging Statistical Network Inference from Large-Scale Gene Expression Data , 2012, PloS one.

[52]  Adriano Velasque Werhli,et al.  Inference of regulatory networks with MCMC sampler guided by mutual information , 2017, SAC.

[53]  Sang C. Suh,et al.  Integrative Gene Regulatory Network inference using multi-omics data , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[54]  Claire Donnat,et al.  Tracking Network Dynamics: a review of distances and similarity metrics , 2018, ArXiv.

[55]  Jagath C. Rajapakse,et al.  Stability of building gene regulatory networks with sparse autoregressive models , 2011, BMC Bioinformatics.

[56]  Cranos M. Williams,et al.  Predicting gene regulatory networks by combining spatial and temporal gene expression data in Arabidopsis root stem cells , 2017, Proceedings of the National Academy of Sciences.

[57]  Dario Floreano,et al.  Generating Realistic In Silico Gene Networks for Performance Assessment of Reverse Engineering Methods , 2009, J. Comput. Biol..

[58]  Hideyuki Suzuki,et al.  Characterizing global evolutions of complex systems via intermediate network representations , 2012, Scientific Reports.

[59]  Torbjörn E. M. Nordling,et al.  GeneSPIDER - gene regulatory network inference benchmarking with controlled network and data properties. , 2017, Molecular bioSystems.

[60]  Melissa J. Davis,et al.  Gene regulatory network inference: evaluation and application to ovarian cancer allows the prioritization of drug targets , 2012, Genome Medicine.

[61]  Ulrich Gerland,et al.  Inference of gene regulation functions from dynamic transcriptome data , 2016, eLife.

[62]  Johan Ugander,et al.  Delay-dependent Stability of Genetic Regulatory Networks , 2008 .

[63]  N. D. Clarke,et al.  Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges , 2010, PloS one.

[64]  Pei Wang,et al.  Integrative random forest for gene regulatory network inference , 2015, Bioinform..

[65]  Andrea Califano,et al.  Integrated biochemical and computational approach identifies BCL6 direct target genes controlling multiple pathways in normal germinal center B cells. , 2008, Blood.

[66]  B. Haibe-Kains,et al.  Gene regulatory networks and their applications: understanding biological and medical problems in terms of networks , 2014, Front. Cell Dev. Biol..

[67]  Masaru Tomita,et al.  Indeterminacy of Reverse Engineering of Gene Regulatory Networks: The Curse of Gene Elasticity , 2007, PloS one.

[68]  A. Barabasi,et al.  Hierarchical Organization of Modularity in Metabolic Networks , 2002, Science.

[69]  Benjamin Haibe-Kains,et al.  Quantitative assessment and validation of network inference methods in bioinformatics , 2014, Front. Genet..

[70]  Muriel Médard,et al.  Network deconvolution as a general method to distinguish direct dependencies in networks , 2013, Nature Biotechnology.

[71]  G. Michailidis,et al.  Autoregressive models for gene regulatory network inference: sparsity, stability and causality issues. , 2013, Mathematical biosciences.

[72]  Cesare Furlanello,et al.  The HIM glocal metric and kernel for network comparison and classification , 2012, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[73]  G. N. Lance,et al.  Mixed-Data Classificatory Programs I - Agglomerative Systems , 1967, Aust. Comput. J..

[74]  Jean Gao,et al.  Integrative approach for inference of gene regulatory networks using lasso-based random featuring and application to psychiatric disorders , 2015, 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[75]  Zongli Lin,et al.  Large scale gene regulatory network inference with a multi-level strategy. , 2016, Molecular bioSystems.

[76]  Alexander S Mikhailov,et al.  Evolutionary reconstruction of networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[77]  Aviv Regev,et al.  Comparative analysis of gene regulatory networks: from network reconstruction to evolution. , 2015, Annual review of cell and developmental biology.

[78]  Frank Emmert-Streib,et al.  Inferring the conservative causal core of gene regulatory networks , 2010, BMC Systems Biology.

[79]  Mads Kærn,et al.  Identifiability and inference of pathway motifs by epistasis analysis. , 2013, Chaos.

[80]  Jigar S Desai,et al.  Improving Gene Regulatory Network Inference by Incorporating Rates of Transcriptional Changes , 2017, Scientific Reports.

[81]  Robin J. Wilson,et al.  An Atlas of Graphs , 1999 .

[82]  S. Horvath,et al.  Statistical Applications in Genetics and Molecular Biology , 2011 .

[83]  Jean-Philippe Vert,et al.  SIRENE: supervised inference of regulatory networks , 2008, ECCB.

[84]  Alberto de la Fuente,et al.  Inferring Gene Networks: Dream or Nightmare? , 2009, Annals of the New York Academy of Sciences.

[85]  Benjamin A. Logsdon,et al.  Gene Expression Network Reconstruction by Convex Feature Selection when Incorporating Genetic Perturbations , 2010, PLoS Comput. Biol..

[86]  Dario Floreano,et al.  GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods , 2011, Bioinform..

[87]  Diogo M. Camacho,et al.  Wisdom of crowds for robust gene network inference , 2012, Nature Methods.

[88]  Giuseppe Jurman,et al.  A Null Model for Pearson Coexpression Networks , 2013, bioRxiv.

[89]  Chong Li,et al.  Linear convergence of CQ algorithms and applications in gene regulatory network inference , 2017 .

[90]  Michael Banf,et al.  Enhancing gene regulatory network inference through data integration with markov random fields , 2017, Scientific Reports.

[91]  Zalmiyah Zakaria,et al.  A review on the computational approaches for gene regulatory network construction , 2014, Comput. Biol. Medicine.

[92]  C J Oates,et al.  Network Inference and Biological Dynamics. , 2011, The annals of applied statistics.

[93]  G. N. Lance,et al.  Computer Programs for Hierarchical Polythetic Classification ("Similarity Analyses") , 1966, Comput. J..

[94]  Delasa Aghamirzaie,et al.  A Machine Learning Approach to Predict Gene Regulatory Networks in Seed Development in Arabidopsis , 2016, Front. Plant Sci..

[95]  Lenwood S. Heath,et al.  PEAK: Integrating Curated and Noisy Prior Knowledge in Gene Regulatory Network Inference , 2017, J. Comput. Biol..

[96]  Gabriel Krouk,et al.  Reverse engineering highlights potential principles of large gene regulatory network design and learning , 2017, npj Systems Biology and Applications.

[97]  Qingshan Jiang,et al.  Gene regulatory network inference using PLS-based methods , 2016, BMC Bioinformatics.

[98]  Cesare Furlanello,et al.  Algebraic stability indicators for ranked lists in molecular profiling , 2008, Bioinform..

[99]  M. Peitsch,et al.  Verification of systems biology research in the age of collaborative competition , 2011, Nature Biotechnology.

[100]  Alessandro Giuliani,et al.  Metabolic pathways variability and sequence/networks comparisons , 2006, BMC Bioinformatics.

[101]  Roozbeh Manshaei,et al.  Sparse and Stable Reconstruction of Genetic Regulatory Networks Using Time Series Gene Expression Data , 2013, BCB.

[102]  W. Kolch,et al.  BGRMI: A method for inferring gene regulatory networks from time-course gene expression data and its application in breast cancer research , 2016, Scientific Reports.

[103]  Cesare Furlanello,et al.  Tumor-infiltrating T lymphocytes improve clinical outcome of therapy-resistant neuroblastoma , 2015, Oncoimmunology.

[104]  Cesare Furlanello,et al.  Stability Indicators in Network Reconstruction , 2012, PloS one.

[105]  Martina Morris,et al.  A statnet Tutorial. , 2008, Journal of statistical software.

[106]  Kenneth N. Brown,et al.  Graph metrics as summary statistics for Approximate Bayesian Computation with application to network model parameter estimation , 2015, J. Complex Networks.

[107]  Korbinian Strimmer,et al.  An empirical Bayes approach to inferring large-scale gene association networks , 2005, Bioinform..

[108]  Veselka Boeva,et al.  A mixture-of-experts approach for gene regulatory network inference , 2016, Int. J. Data Min. Bioinform..

[109]  Korbinian Strimmer,et al.  From correlation to causation networks: a simple approximate learning algorithm and its application to high-dimensional plant gene expression data , 2007, BMC Systems Biology.

[110]  J. Aerts,et al.  SCENIC: Single-cell regulatory network inference and clustering , 2017, Nature Methods.

[111]  Martina Morris,et al.  Specification of Exponential-Family Random Graph Models: Terms and Computational Aspects. , 2008, Journal of statistical software.

[112]  Thibault Espinasse,et al.  Inferring gene regulatory networks from single-cell data: a mechanistic approach , 2017, BMC Systems Biology.

[113]  Kathleen Marchal,et al.  SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms , 2006, BMC Bioinformatics.

[114]  A. Zeng,et al.  An extended transcriptional regulatory network of Escherichia coli and analysis of its hierarchical structure and network motifs. , 2004, Nucleic acids research.

[115]  Ralf Herwig,et al.  IntScore: a web tool for confidence scoring of biological interactions , 2012, Nucleic Acids Res..