On regularized estimation methods for precision and covariance matrix and statistical network inference

Estimation of the covariance matrix is an important problem in statistics in general because the covariance matrix is an essential part of principal component analysis, statistical pattern recognition, multivariate regression and network exploration, just to mention but a few applications. Penalized likelihood methods are used when standard estimates cannot be computed. This is a common case when the number of explanatory variables is much larger compared to the sample size (high-dimensional case). An alternative ridge-type estimator for the precision matrix estimation is introduced in Article I. This estimate is derived using a penalized likelihood estimation method. Undirected networks, which are connected to penalized covariance and precision matrix estimation and some applications related to networks are also explored in this dissertation. In Article II novel statistical methods are used to infer population networks from discrete measurements of genetic data. More precisely, Least Absolute Shrinkage and Selection Operator, LASSO for short, is applied in neighborhood selection. This inferred network is used for more detailed inference of population structures. We illustrate how community detection can be a promising tool in population structure and admixture exploration of genetic data. In addition, in Article IV it is shown how the precision matrix estimator introduced in Article I can be used in graphical model selection via a multiple hypothesis testing procedure. Article III in this dissertation contains a review of current tools for practical graphical model selection and precision/covariance matrix estimation. The other three publications have detailed descriptions of the fundamental computational and mathematical results which create a basis for the methods presented in these articles. Each publication contains a collection of practical research questions where the novel methods can be applied. We hope that these applications will help readers to better understand the possible applications of the methods presented in this dissertation.

[1]  Cun-Hui Zhang,et al.  Sparse matrix inversion with scaled Lasso , 2012, J. Mach. Learn. Res..

[2]  Jarle Tufto,et al.  Genetic variation and structure of house sparrow populations: is there an island effect? , 2013, Molecular ecology.

[3]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[4]  Hongtu Zhu,et al.  The Bayesian Covariance Lasso. , 2013, Statistics and its interface.

[5]  Michael Wolf,et al.  Spectrum Estimation: A Unified Framework for Covariance Matrix Estimation and PCA in Large Dimensions , 2013, J. Multivar. Anal..

[6]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[7]  F. Huang Prediction Error Property of the Lasso Estimator and its Generalization , 2003 .

[8]  Peter Bühlmann,et al.  High-Dimensional Statistics with a View Toward Applications in Biology , 2014 .

[9]  Ming Yuan,et al.  High Dimensional Inverse Covariance Matrix Estimation via Linear Programming , 2010, J. Mach. Learn. Res..

[10]  秀俊 松井,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2014 .

[11]  Tuomas Hämälä Ecological genomics in Arabidopsis lyrata : local adaptation, phenotypic differentiation and reproductive isolation , 2018 .

[12]  Pradeep Ravikumar,et al.  Learning Graphs with a Few Hubs , 2014, ICML.

[13]  K. Huusko,et al.  Dynamics of root-associated fungal communities in relation to disturbance in boreal and subarctic forests , 2018 .

[14]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[15]  Weidong Liu Gaussian graphical model estimation with false discovery rate control , 2013, 1306.0976.

[16]  Steve Horvath,et al.  A Systems Genetics Approach Implicates USF1, FADS3, and Other Causal Candidate Genes for Familial Combined Hyperlipidemia , 2009, PLoS genetics.

[17]  Jiahua Chen,et al.  Extended Bayesian information criteria for model selection with large model spaces , 2008 .

[18]  David I. Warton,et al.  Penalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices , 2008 .

[19]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[20]  Wessel N. van Wieringen,et al.  On the mean squared error of the ridge estimator of the covariance and precision matrix , 2017 .

[21]  Pertti Seppänen Balanced initial teams in early-stage software startups : building a team fitting to the problems and challenges , 2018 .

[22]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[23]  Xing Qiu,et al.  Some Comments on Instability of False Discovery Rate Estimation , 2006, J. Bioinform. Comput. Biol..

[24]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[25]  Pradeep Ravikumar,et al.  BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables , 2013, NIPS.

[26]  Sanni Kinnunen Molecular mechanisms in energy metabolism during seasonal adaptation : aspects relating to AMP-activated protein kinase, key regulator of energy homeostasis , 2018 .

[27]  Olivier Ledoit,et al.  Honey, I Shrunk the Sample Covariance Matrix , 2003 .

[28]  Wessel N. van Wieringen,et al.  Targeted Fused Ridge Estimation of Inverse Covariance Matrices from Multiple High-Dimensional Data Classes , 2015, J. Mach. Learn. Res..

[29]  Guido Giunti,et al.  3MD for Chronic Conditions, a Model for Motivational mHealth Design: Embedded Case Study , 2018, JMIR serious games.

[30]  S. Jackson,et al.  Gene Network Reconstruction by Integration of Prior Biological Knowledge , 2015, G3: Genes, Genomes, Genetics.

[31]  Peter Langfelder,et al.  When Is Hub Gene Selection Better than Standard Meta-Analysis? , 2013, PloS one.

[32]  Larry A. Wasserman,et al.  The huge Package for High-dimensional Undirected Graph Estimation in R , 2012, J. Mach. Learn. Res..

[33]  M. Drton,et al.  Multiple Testing and Error Control in Gaussian Graphical Model Selection , 2005, math/0508267.

[34]  Antti Flyktman,et al.  Effects of transcranial light on molecules regulating circadian rhythm , 2018 .

[35]  R. Dennis Cook,et al.  Cross-Validation of Regression Models , 1984 .

[36]  Pradeep Ravikumar,et al.  QUIC: quadratic approximation for sparse inverse covariance estimation , 2014, J. Mach. Learn. Res..

[37]  Tianxi Cai,et al.  Testing Differential Networks with Applications to Detecting Gene-by-Gene Interactions. , 2015, Biometrika.

[38]  Matthieu Latapy,et al.  Computing Communities in Large Networks Using Random Walks , 2004, J. Graph Algorithms Appl..

[39]  Han Liu,et al.  TIGER : A tuning-insensitive approach for optimally estimating Gaussian graphical models , 2017 .

[40]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[41]  C. O’Brien Statistical Learning with Sparsity: The Lasso and Generalizations , 2016 .

[42]  Rina Foygel,et al.  Extended Bayesian Information Criteria for Gaussian Graphical Models , 2010, NIPS.

[43]  Steve Horvath,et al.  WGCNA: an R package for weighted correlation network analysis , 2008, BMC Bioinformatics.

[44]  Xihong Lin,et al.  The effect of correlation in false discovery rate estimation. , 2011, Biometrika.

[45]  Teemu Tyni,et al.  Direct and inverse scattering problems for perturbations of the biharmonic operator , 2018 .

[46]  Muhammad Zeeshan Asghar,et al.  Remote activity guidance for the elderly utilizing light projection , 2018 .

[47]  Jere Tolvanen,et al.  Informed habitat choice in the heterogeneous world: ecological implications and evolutionary potential , 2018 .

[48]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[49]  Qiang Liu,et al.  Learning Scale Free Networks by Reweighted L1 regularization , 2011, AISTATS.

[50]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[51]  Robert Tibshirani,et al.  Estimating the number of clusters in a data set via the gap statistic , 2000 .

[52]  Weidong Liu,et al.  Fast and adaptive sparse precision matrix estimation in high dimensions , 2012, J. Multivar. Anal..

[53]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[54]  Hao Wang,et al.  Bayesian Graphical Lasso Models and Efficient Posterior Computation , 2012 .

[55]  S. Horvath,et al.  A General Framework for Weighted Gene Co-Expression Network Analysis , 2005, Statistical applications in genetics and molecular biology.

[56]  Kaisa Lehosmaa,et al.  Anthropogenic impacts and restoration of boreal spring ecosystems , 2018 .

[57]  Michael I. Jordan Graphical Models , 1998 .

[58]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[59]  J. Friedman,et al.  New Insights and Faster Computations for the Graphical Lasso , 2011 .

[60]  N. Meinshausen,et al.  Stability selection , 2008, 0809.2932.

[61]  Wessel N. van Wieringen,et al.  Ridge estimation of inverse covariance matrices from high-dimensional data , 2014, Comput. Stat. Data Anal..

[62]  R. Tibshirani,et al.  Sparse estimation of a covariance matrix. , 2011, Biometrika.

[63]  Wei Sun,et al.  Partial correlation matrix estimation using ridge penalty followed by thresholding and re‐estimation , 2014, Biometrics.

[64]  S. Horvath,et al.  Transcriptomic Analysis of Autistic Brain Reveals Convergent Molecular Pathology , 2011, Nature.

[65]  Harrison H. Zhou,et al.  Estimating Sparse Precision Matrix: Optimal Rates of Convergence and Adaptive Estimation , 2012, 1212.2882.

[66]  S. Rosset,et al.  Corrected proof of the result of ‘A prediction error property of the Lasso estimator and its generalization’ by Huang (2003) , 2004 .

[67]  Romain Sarremejane,et al.  Community assembly mechanisms in river networks : exploring the effect of connectivity and disturbances on the assembly of stream communities , 2018 .

[68]  Jianzhi Zhang,et al.  Why Do Hubs Tend to Be Essential in Protein Networks? , 2006, PLoS genetics.

[69]  D. Edwards Introduction to graphical modelling , 1995 .

[70]  Tommi S. Jaakkola,et al.  Inverse Covariance Estimation for High-Dimensional Data in Linear Time and Space: Spectral Methods for Riccati and Sparse Models , 2013, UAI.

[71]  B. Efron Large-Scale Simultaneous Hypothesis Testing , 2004 .

[72]  Michael Oduor Persuasive software design patterns and user perceptions of behaviour change support systems , 2018 .

[73]  Tuija Maliniemi,et al.  Decadal time-scale vegetation changes at high latitudes : responses to climatic and non-climatic drivers , 2018 .

[74]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[75]  S. Horvath,et al.  Gene connectivity, function, and sequence conservation: predictions from modular yeast co-expression networks , 2006, BMC Genomics.

[76]  Santo Fortunato,et al.  Community detection in networks: A user guide , 2016, ArXiv.

[77]  A. Barabasi,et al.  Network biology: understanding the cell's functional organization , 2004, Nature Reviews Genetics.

[78]  Johannes Lederer,et al.  Topology Adaptive Graph Estimation in High Dimensions , 2014, Mathematics.

[79]  Juhani Hopkins The costs and consequences of female sexual signals , 2018 .

[80]  Gábor Csárdi,et al.  The igraph software package for complex network research , 2006 .

[81]  Edward M. Reingold,et al.  Graph drawing by force‐directed placement , 1991, Softw. Pract. Exp..

[82]  Larry A. Wasserman,et al.  Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models , 2010, NIPS.

[83]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[84]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[85]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[86]  Olivier Ledoit,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011, 1207.5322.

[87]  Raya Khanin,et al.  How Scale-Free Are Biological Networks , 2006, J. Comput. Biol..

[88]  Wessel N. van Wieringen,et al.  Application of a New Ridge Estimator of the Inverse Covariance Matrix to the Reconstruction of Gene-Gene Interaction Networks , 2014, CIBB.

[89]  A. Owen Variance of the number of false discoveries , 2005 .

[90]  Santeri Palviainen,et al.  Jaanika Edesi THE EFFECT OF LIGHT SPECTRAL QUALITY ON CRYOPRESERVATION SUCCESS OF POTATO ( SOLANUM TUBEROSUM L . ) SHOOT TIPS IN VITRO , 2018 .

[91]  M. Drton,et al.  Model selection for Gaussian concentration graphs , 2004 .

[92]  Kevin A. Burns,et al.  Genome evolution in the allotetraploid frog Xenopus laevis , 2016, Nature.