Maximum entropy property of discrete-time stable spline kernel

In this paper, the maximum entropy property of the discrete-time first-order stable spline kernel is studied. The advantages of studying this property in discrete-time domain instead of continuous-time domain are outlined. One of such advantages is that the differential entropy rate is well-defined for discrete-time stochastic processes. By formulating the maximum entropy problem for discrete-time stochastic processes we provide a simple and self-contained proof to show what maximum entropy property the discrete-time first-order stable spline kernel has.

[1]  Alessandro Chiuso,et al.  Prediction error identification of linear systems: A nonparametric Gaussian regression approach , 2011, Autom..

[2]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[3]  L. Arnold.  Stochastic Differential Equations: Theory and Applications , 1992 .

[4]  Lennart Ljung,et al.  System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques , 2014, IEEE Transactions on Automatic Control.

[5]  John G. van Bosse,et al.  Wiley Series in Telecommunications and Signal Processing , 2006 .

[6]  Henrik Ohlsson,et al.  Four Encounters with System Identification , 2011, Eur. J. Control.

[7]  E. Jaynes.  On the rationale of maximum-entropy methods , 1982, Proceedings of the IEEE.

[8]  Henrik Ohlsson,et al.  On the estimation of transfer functions, regularizations and Gaussian processes - Revisited , 2012, Autom..

[9]  G. Ferrari-Trecate,et al.  MAXENT priors for stochastic filtering problems , 1998 .

[10]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[11]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[12]  L. Ljung,et al.  On the Estimation of Transfer Functions, Regularizations and Gaussian Processes – Revisited , 2011 .

[13]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[14]  Giuseppe De Nicolao,et al.  Kernel selection in linear system identification Part I: A Gaussian process perspective , 2011, IEEE Conference on Decision and Control and European Control Conference.