Efficient Recursive Implementation of Spatial-Temporal Gaussian Process Regression

The current implementation of the spatial-temporal Gaussian process regression has computational complexity O(NM3), where N and M are the number of temporal and spatial data, respectively, and thus can only be applied to data with large N but relatively small M. In this work, we show that by exploring the Kronecker structure in the state-space model realization of the spatial-temporal Gaussian process, we can extend the current implementation with a coordinate transformation and an output transformation (corresponding to data preprocessing), such that the computational complexity is reduced to O(M3 +NM2 +NM) and therefore the proposed implementation can be applied to data with large N and moderately large M. Moreover, the proposed implementation can be parallelized and the computational complexity can be further lowered if parallel computing is adopted.

[1]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[2]  C. Striebel,et al.  On the maximum likelihood estimates for linear dynamic systems , 1965 .

[3]  Henrik Ohlsson,et al.  On the estimation of transfer functions, regularizations and Gaussian processes - Revisited , 2012, Autom..

[4]  Lennart Ljung,et al.  Implementation of algorithms for tuning parameters in regularized least squares problems in system identification , 2013, Autom..

[5]  A. Karpatne,et al.  Spatio-Temporal Data Mining: A Survey of Problems and Methods , 2017, ArXiv.

[6]  Ruggero Carli,et al.  Efficient Spatio-Temporal Gaussian Regression via Kalman Filtering , 2017, Autom..

[7]  Arno Solin,et al.  Spatio-Temporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing , 2013 .

[8]  Lennart Ljung,et al.  Asymptotic Properties of Hyperparameter Estimators by Using Cross-Validations for Regularized System Identification , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[9]  Tianshi Chen,et al.  On kernel design for regularized LTI system identification , 2016, Autom..

[10]  Shuguang Cui,et al.  Distributed Gaussian Processes Hyperparameter Optimization for Big Data Using Proximal ADMM , 2019, IEEE Signal Processing Letters.

[11]  J. Møller,et al.  Handbook of Spatial Statistics , 2008 .

[12]  Giulio Bottegal,et al.  The generalized cross validation filter , 2017, Autom..

[13]  Feng Yin,et al.  Recursive Implementation of Gaussian Process Regression for Spatial-Temporal Data Modeling , 2019, 2019 11th International Conference on Wireless Communications and Signal Processing (WCSP).

[14]  Jouni Hartikainen,et al.  Kalman filtering and smoothing solutions to temporal Gaussian process regression models , 2010, 2010 IEEE International Workshop on Machine Learning for Signal Processing.

[15]  Lennart Ljung,et al.  On Asymptotic Properties of Hyperparameter Estimators for Kernel-based Regularization Methods , 2017, Autom..

[16]  Nikos Pelekis,et al.  Literature review of spatio-temporal database models , 2004, The Knowledge Engineering Review.

[17]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[18]  Ruggero Carli,et al.  Machine learning meets Kalman Filtering , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).