A tutorial on Gaussian process regression with a focus on exploration-exploitation scenarios

This tutorial introduces the reader to Gaussian process regression as a tool to model, actively explore and exploit unknown functions. Gaussian process regression is a powerful, non-parametric Bayesian approach towards regression problems that can be utilized in exploration and exploitation scenarios. This tutorial aims to provide an accessible introduction to these techniques. We will introduce Gaussian processes as a distribution over functions used for Bayesian non-parametric regression and demonstrate different applications of it. Didactic examples will include a simple regression problem, a demonstration of kernel-encoded prior assumptions, a pure exploration scenario within an optimal design framework, and a bandit-like exploration-exploitation scenario where the goal is to recommend movies. Beyond that, we describe a situation in which an additional constraint (not to sample below a certain threshold) needs to be accounted for and summarize recent psychological experiments utilizing Gaussian processes. Software and literature pointers will be provided.