A Survey on Optimal Transport for Machine Learning: Theory and Applications

Optimal Transport (OT) theory has seen an increasing amount of attention from the computer science community due to its potency and relevance in modeling and machine learning. It introduces means that serve as powerful ways to compare probability distributions with each other, as well as producing optimal mappings to minimize cost functions. Therefor, it has been deployed in computer vision, improving image retrieval, image interpolation, and semantic correspondence algorithms, as well as other fields such as domain adaptation, natural language processing, and variational inference. In this survey, we propose to convey the emerging promises of the optimal transport methods across various fields, as well as future directions of study for OT in machine learning. We will begin by looking at the history of optimal transport and introducing the founders of this field. We then give a brief glance into the algorithms related to OT. Then, we will follow up with a mathematical formulation and the prerequisites to understand OT, these include Kantorovich duality, entropic regularization, KL Divergence, and Wassertein barycenters. Since OT is a computationally expensive problem, we then introduce the entropy-regularized version of computing optimal mappings, which allowed OT problems to become applicable in a wide range of machine learning problems. In fact, the methods generated from OT theory are competitive with the current state-of-the-art methods. The last portion of this survey will analyze papers that focus on the application of OT within the context of machine learning. We first cover computer vision problems; these include GANs, semantic correspondence, and convolutional Wasserstein distances. Furthermore, we follow this up by breaking down research papers that focus on graph learning, neural architecture search, document representation, and domain adaptation. We close the paper with a small section on future research. Of the recommendations presented, three main problems are fundamental to allow OT to become widely applicable but rely strongly on its mathematical formulation and thus are hardest to answer. Since OT is a novel method, there is plenty of space for new research, and with more and more competitive methods (either on an accuracy level or computational speed level) being created, the future of applied optimal transport is bright as it has become pervasive in machine learning.