Bayesian Alignments of Warped Multi-Output Gaussian Processes

We propose a novel Bayesian approach to modelling nonlinear alignments of time series based on latent shared information. We apply the method to the real-world problem of finding common structure in the sensor data of wind turbines introduced by the underlying latent and turbulent wind field. The proposed model allows for both arbitrary alignments of the inputs and non-parametric output warpings to transform the observations. This gives rise to multiple deep Gaussian process models connected via latent generating processes. We present an efficient variational approximation based on nested variational compression and show how the model can be used to extract shared information between dependent time series, recovering an interpretable functional decomposition of the learning problem. We show results for an artificial data set and real-world data of two wind turbines.

[1]  Jasper Snoek,et al.  Input Warping for Bayesian Optimization of Non-Stationary Functions , 2014, ICML.

[2]  Timothy C. Coburn,et al.  Geostatistics for Natural Resources Evaluation , 2000, Technometrics.

[3]  Neil D. Lawrence,et al.  Efficient Multioutput Gaussian Processes through Variational Inducing Kernels , 2010, AISTATS.

[4]  Marc Peter Deisenroth,et al.  Doubly Stochastic Variational Inference for Deep Gaussian Processes , 2017, NIPS.

[5]  J G Schepers,et al.  Improved modelling of wake aerodynamics and assessment of new farm control strategies , 2007 .

[6]  Neil D. Lawrence,et al.  Sparse Convolved Gaussian Processes for Multi-output Regression , 2008, NIPS.

[7]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[8]  Fernando De la Torre,et al.  Generalized time warping for multi-modal alignment of human motion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Neil D. Lawrence,et al.  Nested Variational Compression in Deep Gaussian Processes , 2014, 1412.1370.

[10]  Marcus R. Frean,et al.  Dependent Gaussian Processes , 2004, NIPS.

[11]  Miguel Lázaro-Gredilla,et al.  Bayesian Warped Gaussian Processes , 2012, NIPS.

[12]  James Hensman,et al.  Scalable Variational Gaussian Process Classification , 2014, AISTATS.

[13]  Eilyan Bitar,et al.  Coordinated control of a wind turbine array for power maximization , 2013, 2013 American Control Conference.

[14]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[15]  Maryam Soleimanzadeh,et al.  Controller design for a wind farm, considering both power and load aspects , 2011 .

[16]  Carl E. Rasmussen,et al.  Warped Gaussian Processes , 2003, NIPS.

[17]  Neil D. Lawrence,et al.  Bayesian Gaussian Process Latent Variable Model , 2010, AISTATS.

[18]  Alexis Boukouvalas,et al.  GPflow: A Gaussian Process Library using TensorFlow , 2016, J. Mach. Learn. Res..

[19]  Ryan P. Adams,et al.  Avoiding pathologies in very deep networks , 2014, AISTATS.

[20]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[21]  Neil D. Lawrence,et al.  Gaussian Processes for Big Data , 2013, UAI.

[22]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.