Efficient parameter identification and model selection in nonlinear dynamical systems via sparse Bayesian learning

Bayesian inference plays a central role in many of today's system identification algorithms. However, one of its major drawbacks is that it often requires solutions to analytically intractable integrals, and this has led to solutions via Markov Chain Monte Carlo (MCMC). Today, few Bayesian system identification methods can compete with the robustness of the family of MCMC solutions. However, MCMC suffers severely from its computational cost. Partly fuelled by the field of Compressive Sensing (CS), an interest in the machine learning community has arisen in sparse linear regression. Its value in identification of dynamical systems has also recently started to receive some attention. The idea is to represent the range of possible candidate functional forms that have generated a specific data set using a dictionary, and to then apply standard Lasso regression to select the "best" basis, with sparsity constraints. A major problem in this approach is that Lasso regression (and its derivatives) requires tuning of the regularisation parameter. In this paper, a procedure for selecting the "best basis" (and thus performing both model selection and system identification) is presented through a sparse Bayesian learning approach. Sparsity is induced via a hierarchical Gaussian prior and an approximation to the posterior distribution is sought using an iterative optimisation scheme for finding the optimal hyper-priors that govern prior and hence, the level of sparsity in the solution. The method is applied to five systems of engineering interest, which include a baseline linear system, an additive quadratic damping term, cubic stiffness (Duffing oscillator), Coulomb damping and a Bouc-Wen hysteresis model. The results are shown using numerical simulations. It is shown that this approach can identify not only the correct model parameters, but whether a nonlinearity is present in the system as well its type. With the formulation being Bayesian, it also yields estimates of uncertainty over the selected basis functions and predicted responses.