Network Reconstruction From High-Dimensional Ordinary Differential Equations

ABSTRACT We consider the task of learning a dynamical system from high-dimensional time-course data. For instance, we might wish to estimate a gene regulatory network from gene expression data measured at discrete time points. We model the dynamical system nonparametrically as a system of additive ordinary differential equations. Most existing methods for parameter estimation in ordinary differential equations estimate the derivatives from noisy observations. This is known to be challenging and inefficient. We propose a novel approach that does not involve derivative estimation. We show that the proposed method can consistently recover the true network structure even in high dimensions, and we demonstrate empirical improvement over competing approaches. Supplementary materials for this article are available online.

[1]  Dario Floreano,et al.  GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods , 2011, Bioinform..

[2]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[3]  A. Tsybakov,et al.  Sparse recovery under matrix uncertainty , 2008, 0812.2818.

[4]  Po-Ling Loh,et al.  High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity , 2011, NIPS.

[5]  G. DeAngelis,et al.  Parallel Input Channels to Mouse Primary Visual Cortex , 2010, The Journal of Neuroscience.

[6]  Hulin Wu,et al.  Sieve Estimation of Constant and Time-Varying Coefficients in Nonlinear Ordinary Differential Equation Models by Considering Both Numerical Error and Measurement Error. , 2010, Annals of statistics.

[7]  R. Tibshirani,et al.  Linear Smoothers and Additive Models , 1989 .

[8]  Yingcun Xia,et al.  Asymptotic Behavior of Bandwidth Selected by the Cross-Validation Method for Local Polynomial Fitting , 2002 .

[9]  M. Wells,et al.  Variations and Fluctuations of the Number of Individuals in Animal Species living together , 2006 .

[10]  D. Floreano,et al.  Revealing strengths and weaknesses of methods for gene network inference , 2010, Proceedings of the National Academy of Sciences.

[11]  Dario Floreano,et al.  Generating Realistic In Silico Gene Networks for Performance Assessment of Reverse Engineering Methods , 2009, J. Comput. Biol..

[12]  Hulin Wu,et al.  Statistical methods for HIV dynamic studies in AIDS clinical trials , 2005, Statistical methods in medical research.

[13]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[14]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[15]  Yanyuan Ma,et al.  Quick and easy one‐step parameter estimation in differential equations , 2014 .

[16]  Lorenz T. Biegler,et al.  Nonlinear parameter estimation: A case study comparison , 1986 .

[17]  Jiguo Cao,et al.  On the selection of ordinary differential equation models with application to predator‐prey dynamical models , 2015, Biometrics.

[18]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[19]  Jonathan E. Taylor,et al.  On model selection consistency of penalized M-estimators: a geometric theory , 2013, NIPS.

[20]  J. Lafferty,et al.  Sparse additive models , 2007, 0711.4555.

[21]  J. Varah A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations , 1982 .

[22]  M. Benson,et al.  Parameter fitting in dynamic models , 1979 .

[23]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[24]  Tso-Jung Yen,et al.  Discussion on "Stability Selection" by Meinshausen and Buhlmann , 2010 .

[25]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[26]  Xiaohua Xia,et al.  On Identifiability of Nonlinear ODE Models and Applications in Viral Dynamics , 2011, SIAM Rev..

[27]  Chenlei Leng,et al.  Unified LASSO Estimation by Least Squares Approximation , 2007 .

[28]  L Wang,et al.  Robust Estimation for Ordinary Differential Equation Models , 2011, Biometrics.

[29]  Runze Li,et al.  Variable Selection in Measurement Error Models. , 2010, Bernoulli : official journal of the Bernoulli Society for Mathematical Statistics and Probability.

[30]  Hongyu Zhao,et al.  Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations , 2009, 0903.3400.

[31]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[32]  G. Michailidis,et al.  Network Reconstruction Using Nonparametric Additive ODE Models , 2014, PloS one.

[33]  W. Lim,et al.  Defining Network Topologies that Can Achieve Biochemical Adaptation , 2009, Cell.

[34]  Itai Dattner,et al.  Optimal rate of direct estimators in systems of ordinary differential equations linear in functions of the parameters , 2013 .

[35]  I. Chou,et al.  Recent developments in parameter estimation and structure identification of biochemical and genomic systems. , 2009, Mathematical biosciences.

[36]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[37]  N. Meinshausen,et al.  Stability selection , 2008, 0809.2932.

[38]  Volker Roth,et al.  The Group-Lasso for generalized linear models: uniqueness of solutions and efficient algorithms , 2008, ICML '08.

[39]  Hulin Wu,et al.  Sparse Additive Ordinary Differential Equations for Dynamic Gene Regulatory Network Modeling , 2014, Journal of the American Statistical Association.

[40]  Jiguo Cao,et al.  Parameter estimation for differential equations: a generalized smoothing approach , 2007 .

[41]  Hulin Wu,et al.  Parameter Estimation for Differential Equation Models Using a Framework of Measurement Error in Regression Models , 2008, Journal of the American Statistical Association.

[42]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[43]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[44]  Quentin Clairon,et al.  Parametric Estimation of Ordinary Differential Equations With Orthogonality Conditions , 2014, Journal of the American Statistical Association.

[45]  M. Wand Local Regression and Likelihood , 2001 .

[46]  Jiguo Cao,et al.  Estimating dynamic models for gene regulation networks , 2008, Bioinform..

[47]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[48]  Hongzhe Li,et al.  High-Dimensional ODEs Coupled With Mixed-Effects Modeling Techniques for Dynamic Gene Regulatory Network Identification , 2011, Journal of the American Statistical Association.

[49]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[50]  Chris A. J. Klaassen,et al.  √n-consistent parameter estimation for systems of ordinary differential equations : bypassing numerical integration via smoothing , 2010, 1007.3880.

[51]  Guohua Pan,et al.  Local Regression and Likelihood , 1999, Technometrics.

[52]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[53]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[54]  S. Ellner,et al.  Fitting population dynamic models to time-series data by gradient matching , 2002 .

[55]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[56]  H. Weyl Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung) , 1912 .

[57]  Gábor Lugosi,et al.  Concentration Inequalities , 2008, COLT.

[58]  Robert Tibshirani,et al.  STANDARDIZATION AND THE GROUP LASSO PENALTY. , 2012, Statistica Sinica.

[59]  N. Brunel Parameter estimation of ODE’s via nonparametric estimators , 2007, 0710.4190.

[60]  Ali Shojaie,et al.  Graph Estimation with Joint Additive Models. , 2013, Biometrika.

[61]  C. Sbardella High dimensional regression , 2011 .