Consolidating the Meta-Learning Zoo: A Unifying Perspective as Posterior Predictive Inference

A plethora of methods and approaches combining meta-learning [19, 21] with deep neural networks have recently been proposed, achieving great success in applications such as few-shot learning. Much of the existing work may be characterized as either gradient-based [4, 17, 9], metric-based [22, 20], or amortized MAP based meta-learning [8, 16]. Due to the ubiquity of recent work, a unifying view is useful for understanding and improving these methods. Existing frameworks [5, 9] are limited to specific families of approaches, namely gradient-based methods [17, 4]. In this paper we develop a framework for meta-learning approximate probabilistic inference for prediction, or ML-PIP for short. ML-PIP provides this unifying perspective in terms of amortizing posterior predictive distributions. We show that ML-PIP re-frames and extends existing probabilistic interpretations of meta-learning [5, 9] to cover both point-estimates and variational posteriors, as well as a broader class of methods, including gradient based meta-learning [4, 17], metric based meta-learning [20], amortized MAP inference [16], and conditional probability modelling [6, 7].