The quest for the right kernel in Bayesian impulse response identification: The use of OBFs

Kernel-based regularization approaches for impulse response estimation of Linear Time-Invariant (LTI) systems have received a lot of attention recently. The reason is that regularized least-squares estimators may achieve a favorable bias/variance trade-off compared with classical Prediction Error Minimization (PEM) methods. To fully exploit this property, the kernel function needs to capture relevant aspects of the data-generating system at hand. Hence, it is important to design automatic procedures for kernel design based on data or prior knowledge. The kernel models, so far introduced, focus on encoding smoothness and BIBO-stability of the expected impulse response while other properties, like oscillatory behavior or the presence of fast and slow poles, have not been successfully implemented in kernel design. Inspired by the representation theory of dynamical systems, we show how to build stable kernels that are able to capture particular aspects of system dynamics via the use of Orthonormal Basis Functions (OBFs). In particular, desired dynamic properties can be easily encoded via the generating poles of OBFs. Such poles are seen as hyperparameters which are tuned via marginal likelihood optimization. Special cases of our kernel construction include Laguerre, Kautz, and Generalized OBFs (GOBFs)-based kernel structures. Monte-Carlo simulations show that the OBFs-based kernels perform well compared with stable spline/TC kernels, especially for slow systems with dominant poles close to the unit circle. Moreover, the capability of Kautz basis to model resonating systems is also shown.