Parallelizable sparse inverse formulation Gaussian processes (SpInGP)

We propose a parallelizable sparse inverse formulation Gaussian process (SpInGP) for temporal models. It uses a sparse precision GP formulation and sparse matrix routines to speed up the computations. Due to the state-space formulation used in the algorithm, the time complexity of the basic SpInGP is linear, and because all the computations are parallelizable, the parallel form of the algorithm is sublinear in the number of data points. We provide example algorithms to implement the sparse matrix routines and experimentally test the method using both simulated and real data.

[1]  Leslie Greengard,et al.  Fast Direct Methods for Gaussian Processes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[3]  Vickie E. Lynch,et al.  BCYCLIC: A parallel block tridiagonal matrix cyclic solver , 2010, J. Comput. Phys..

[4]  Mathias Jacquelin,et al.  PSelInv—A Distributed Memory Parallel Algorithm for Selected Inversion , 2017, ACM Trans. Math. Softw..

[5]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[6]  J. Vanhatalo,et al.  Approximate inference for disease mapping with sparse Gaussian processes , 2010, Statistics in medicine.

[7]  Juha Karhunen,et al.  Gaussian Process kernels for popular state-space time series models , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[8]  Mihai Anitescu,et al.  Real-Time Stochastic Optimization of Complex Energy Systems on High-Performance Computers , 2014, Computing in Science & Engineering.

[9]  Simo Särkkä,et al.  Bayesian Filtering and Smoothing , 2013, Institute of Mathematical Statistics textbooks.

[10]  Arno Solin,et al.  Explicit Link Between Periodic Covariance Functions and State Space Models , 2014, AISTATS.

[11]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[12]  Jouni Hartikainen,et al.  Kalman filtering and smoothing solutions to temporal Gaussian process regression models , 2010, 2010 IEEE International Workshop on Machine Learning for Signal Processing.

[13]  Eric Darve,et al.  A hybrid method for the parallel computation of Green's functions , 2009, J. Comput. Phys..

[14]  S Roberts,et al.  Gaussian processes for time-series modelling , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[15]  Simo Srkk,et al.  Bayesian Filtering and Smoothing , 2013 .

[16]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[17]  พงศ์ศักดิ์ บินสมประสงค์,et al.  FORMATION OF A SPARSE BUS IMPEDANCE MATRIX AND ITS APPLICATION TO SHORT CIRCUIT STUDY , 1980 .

[18]  Simo Särkkä,et al.  Batch nonlinear continuous-time trajectory estimation as exactly sparse Gaussian process regression , 2014, Autonomous Robots.

[19]  Roger L. Davis,et al.  GPGPU parallel algorithms for structured-grid CFD codes , 2011 .

[20]  Stephen J. Roberts,et al.  Gaussian Processes for Timeseries Modelling. , 2012 .