Variational Inference for Linear Systems with Latent Parameter Space

We present a method to perform identification of systems with external inputs whose parameters are indexed by a lower-dimensional latent space. We apply a variational Bayes inference method to approximate the posterior distribution of the system parameters and latent variables, given input and output measurements. This approach seeks to minimize the Kullback-Leibler divergence between the full (but intractable) posterior distribution of the parameters and an approximating (yet tractable) factorized distribution. The method enables inference for systems whose parameters are subject to latent sources of variation, and therefore constitutes a relevant tool for modeling and control in complex domains, such as biological systems and neuroscience.

[1]  Michael I. Jordan,et al.  Variational Bayesian Inference with Stochastic Search , 2012, ICML.

[2]  Roland Toth,et al.  Modeling and Identification of Linear Parameter-Varying Systems , 2010 .

[3]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[4]  Maciej Niedzwiecki,et al.  Identification of Time-Varying Processes , 2000 .

[5]  Victor M. Preciado,et al.  Identification of State-space Linear Time-varying Systems with Sum-of-norms Regularization , 2018, 2018 Annual American Control Conference (ACC).

[6]  Christian Hoffmann,et al.  A Survey of Linear Parameter-Varying Control Applications Validated by Experiments or High-Fidelity Simulations , 2015, IEEE Transactions on Control Systems Technology.

[7]  Boris I. Godoy,et al.  An EM-based identification algorithm for a class of hybrid systems with application to power electronics , 2014, Int. J. Control.

[8]  Tapani Raiko,et al.  Linear State-Space Model with Time-Varying Dynamics , 2014, ECML/PKDD.

[9]  Javad Mohammadpour,et al.  A Bayesian approach for model identification of LPV systems with uncertain scheduling variables , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[10]  Roland Tóth,et al.  LPV State-space model identification in the Bayesian setting: A 3-step procedure , 2016, 2016 American Control Conference (ACC).

[11]  Wei Wu,et al.  A new look at state-space models for neural data , 2010, Journal of Computational Neuroscience.

[12]  Tohru Katayama,et al.  Subspace Methods for System Identification , 2005 .

[13]  Javad Mohammadpour,et al.  A Bayesian Approach for LPV Model Identification and Its Application to Complex Processes , 2017, IEEE Transactions on Control Systems Technology.

[14]  P.M.J. Van den Hof,et al.  Modeling and Identification of Linear Parameter-Varying Systems, an Orthonormal Basis Function Approach , 2004 .

[15]  Sean R. Anderson,et al.  Sparse Bayesian Nonlinear System Identification Using Variational Inference , 2018, IEEE Transactions on Automatic Control.

[16]  V. Verdult.  Non linear system identification : a state-space approach , 2002 .

[17]  Roland Tóth,et al.  LPV system identification under noise corrupted scheduling and output signal observations , 2015, Autom..

[18]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[19]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[20]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[21]  J. W. van Wingerden,et al.  A kernel based approach for LPV subspace identification , 2015 .

[22]  Henri Bourlès,et al.  Linear Time-Varying Systems: Algebraic-Analytic Approach , 2011 .

[23]  Brett Ninness,et al.  Robust maximum-likelihood estimation of multivariable dynamic systems , 2005, Autom..

[24]  Jongeun Choi,et al.  Linear Parameter-Varying Control for Engineering Applications , 2013, Springer Briefs in Electrical and Computer Engineering.

[25]  Javad Mohammadpour,et al.  A Kernel-based Approach to MIMO LPV State-space Identification and Application to a Nonlinear Process System , 2015 .

[26]  Javad Mohammadpour,et al.  A Bayesian approach for estimation of linear-regression LPV models , 2014, 53rd IEEE Conference on Decision and Control.

[27]  Zhe Chen,et al.  Advanced state space methods for neural and clinical data , 2015 .

[28]  B. Ninness,et al.  System identification of linear parameter varying state-space models , 2011 .

[29]  Yong Zeng,et al.  State-space models : applications in economics and finance , 2013 .

[30]  Christian Hoffmann,et al.  A factor graph approach to parameter identification for affine LPV systems , 2017, 2017 American Control Conference (ACC).