A Block Coordinate Descent Proximal Method for Simultaneous Filtering and Parameter Estimation

We propose and analyze a block coordinate descent proximal algorithm (BCD-prox) for simultaneous filtering and parameter estimation of ODE models. As we show on ODE systems with up to d=40 dimensions, as compared to state-of-the-art methods, BCD-prox exhibits increased robustness (to noise, parameter initialization, and hyperparameters), decreased training times, and improved accuracy of both filtered states and estimated parameters. We show how BCD-prox can be used with multistep numerical discretizations, and we establish convergence of BCD-prox under hypotheses that include real systems of interest.

[1]  M. Benson,et al.  Parameter fitting in dynamic models , 1979 .

[2]  Charles L. Byrne,et al.  Alternating Minimization, Proximal Minimization and Optimization Transfer Are Equivalent , 2015, 1512.03034.

[3]  R. Palais,et al.  Differential Equations, Mechanics, and Computation , 2012 .

[4]  J. NAGUMOt,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 2006 .

[5]  Mark A. Girolami,et al.  Bayesian inference for differential equations , 2008, Theor. Comput. Sci..

[6]  Jinbo Bi,et al.  Multi-view Sparse Co-clustering via Proximal Alternating Linearized Minimization , 2015, ICML.

[7]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[8]  C. R. Jones,et al.  Determination of Rate Constants for Complex Kinetics Models , 1967 .

[9]  Charles L. Byrne,et al.  Alternating Minimization as Sequential Unconstrained Minimization: A Survey , 2012, Journal of Optimization Theory and Applications.

[10]  R. Barber,et al.  Alternating minimization and alternating descent over nonconvex sets , 2017, 1709.04451.

[11]  Yuanzhi Li,et al.  Recovery guarantee of weighted low-rank approximation via alternating minimization , 2016, ICML.

[12]  임규호,et al.  Optimal sites for supplementary weather observations , 2011 .

[13]  Itai Dattner,et al.  Optimal rate of direct estimators in systems of ordinary differential equations linear in functions of the parameters , 2013 .

[14]  Luigi Grippo,et al.  On the convergence of the block nonlinear Gauss-Seidel method under convex constraints , 2000, Oper. Res. Lett..

[15]  K. Emanuel,et al.  Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model , 1998 .

[16]  Constantine Caramanis,et al.  Alternating Minimization for Mixed Linear Regression , 2013, ICML.

[17]  Chris A. J. Klaassen,et al.  √n-consistent parameter estimation for systems of ordinary differential equations : bypassing numerical integration via smoothing , 2010, 1007.3880.

[18]  Neil D. Lawrence,et al.  Accelerating Bayesian Inference over Nonlinear Differential Equations with Gaussian Processes , 2008, NIPS.

[19]  P. James McLellan,et al.  Parameter estimation in continuous-time dynamic models using principal differential analysis , 2006, Comput. Chem. Eng..

[20]  Peter L. Bartlett,et al.  Alternating minimization for dictionary learning with random initialization , 2017, NIPS.

[21]  Pramod K. Varshney,et al.  Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization , 2017, ICML.

[22]  R. FitzHugh.  Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[23]  Yonathan Bard,et al.  Nonlinear parameter estimation , 1974 .

[24]  Dirk Husmeier,et al.  ODE parameter inference using adaptive gradient matching with Gaussian processes , 2013, AISTATS.

[25]  J. Varah.  A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations , 1982 .

[26]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[27]  J Kurths,et al.  Estimation of parameters and unobserved components for nonlinear systems from noisy time series. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  Stefan Bauer,et al.  Scalable Variational Inference for Dynamical Systems , 2017, NIPS.

[29]  L Wang,et al.  Robust Estimation for Ordinary Differential Equation Models , 2011, Biometrics.

[30]  L. Hosten,et al.  A comparative study of short cut procedures for parameter estimation in differential equations , 1979 .

[31]  T. Brubaker,et al.  Nonlinear Parameter Estimation , 1979 .

[32]  Jiguo Cao,et al.  Estimating dynamic models for gene regulation networks , 2008, Bioinform..

[33]  Ziming Zhang,et al.  Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks , 2017, NIPS.

[34]  Hulin Wu,et al.  Parameter Estimation for Differential Equation Models Using a Framework of Measurement Error in Regression Models , 2008, Journal of the American Statistical Association.

[35]  M. Newville,et al.  Lmfit: Non-Linear Least-Square Minimization and Curve-Fitting for Python , 2014 .

[36]  Alfred J. Lotka,et al.  The growth of mixed populations: Two species competing for a common food supply , 1978 .

[37]  O. Rössler.  An equation for continuous chaos , 1976 .

[38]  Jiguo Cao,et al.  Parameter estimation for differential equations: a generalized smoothing approach , 2007 .