Functional Optimal Transport: Mapping Estimation and Domain Adaptation for Functional data

Optimal transport (OT) has generated much recent interest by its capability of finding mappings that transport mass from one distribution to another, and found useful roles in machine learning tasks such as unsupervised learning, domain adaptation and transfer learning. On the other hand, in many applications data are generated by complex mechanisms involving convoluted spaces of functions, curves and surfaces in high dimensions. Functional data analysis provides a useful framework of treatment for such domains. In this paper we introduce a novel formulation of optimal transport problem in functional spaces and develop an efficient learning algorithm for finding the stochastic map between functional domains. We apply our method to synthetic datasets and study the geometric properties of the transport map. Experiments on real-world datasets of robot arm trajectories and digit numbers further demonstrate the effectiveness of our method on applications of domain adaptation and generative modeling.

[1]  Ransalu Senanayake,et al.  Online Domain Adaptation for Occupancy Mapping , 2020, Robotics: Science and Systems.

[2]  C. Karen Liu,et al.  Assistive Gym: A Physics Simulation Framework for Assistive Robotics , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Nicolas Courty,et al.  Mapping Estimation for Discrete Optimal Transport , 2016, NIPS.

[4]  Bernardete Ribeiro,et al.  Gaussian Process Classification and Active Learning with Multiple Annotators , 2014, ICML.

[5]  Emily B. Fox,et al.  Representing and Denoising Wearable ECG Recordings , 2020, ArXiv.

[6]  Tommi S. Jaakkola,et al.  Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces , 2020, AISTATS.

[7]  Asuka Takatsu.  Wasserstein geometry of Gaussian measures , 2011 .

[8]  Dinh Q. Phung,et al.  Multilevel Clustering via Wasserstein Means , 2017, ICML.

[9]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[10]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Matthew Reimherr,et al.  Formal Privacy for Functional Data with Gaussian Perturbations , 2017, ICML.

[12]  Wenhao Ding,et al.  Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes , 2020, NeurIPS.

[13]  L. Kantorovich.  On the Translocation of Masses , 2006 .

[14]  Han Zhang,et al.  Improving GANs Using Optimal Transport , 2018, ICLR.

[15]  Limin Wang,et al.  Karhunen-Loeve expansions and their applications , 2008 .

[16]  Marc Toussaint,et al.  Trajectory prediction: learning to map situations to robot trajectories , 2009, ICML '09.

[17]  M. Gelbrich.  On a Formula for the L2 Wasserstein Metric between Measures on Euclidean and Hilbert Spaces , 1990 .

[18]  Cheng Meng,et al.  Large-scale optimal transport map estimation using projection pursuit , 2021, NeurIPS.

[19]  Piotr Kokoszka,et al.  Inference for Functional Data with Applications , 2012 .

[20]  T. Hsing,et al.  Theoretical foundations of functional data analysis, with an introduction to linear operators , 2015 .

[21]  Sewoong Oh,et al.  Optimal transport mapping via input convex neural networks , 2019, ICML.

[22]  Abhinav Gupta,et al.  Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation , 2018, CoRL.

[23]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[25]  Nicolas Courty,et al.  Large Scale Optimal Transport and Mapping Estimation , 2017, ICLR.

[26]  Tommi S. Jaakkola,et al.  Towards Optimal Transport with Global Invariances , 2018, AISTATS.

[27]  C. Fletcher.  Computational Galerkin Methods , 1983 .

[28]  Hongyuan Zha,et al.  On Scalable and Efficient Computation of Large Scale Optimal Transport , 2019, ICML.

[29]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[30]  F. Demengel,et al.  Functional Spaces for the Theory of Elliptic Partial Differential Equations , 2012 .

[31]  Aaron Courville,et al.  Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization , 2021, ICLR.

[32]  Jing Lei.  Convergence and concentration of empirical measures under Wasserstein distance in unbounded functional spaces , 2018, Bernoulli.

[33]  Aasa Feragen,et al.  Learning from uncertain curves: The 2-Wasserstein metric for Gaussian processes , 2017, NIPS.

[34]  Sergey Levine,et al.  Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[36]  Gabriel Peyré,et al.  Stochastic Optimization for Large-scale Optimal Transport , 2016, NIPS.

[37]  Silvio Savarese,et al.  ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation , 2018, CoRL.

[38]  Meng Zhang,et al.  Earth Mover’s Distance Minimization for Unsupervised Bilingual Lexicon Induction , 2017, EMNLP.

[39]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[40]  Nicolas Courty,et al.  DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation , 2018, ECCV.

[41]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[42]  Edouard Grave,et al.  Unsupervised Alignment of Embeddings with Wasserstein Procrustes , 2018, AISTATS.

[43]  C. Villani.  Optimal Transport: Old and New , 2008 .

[44]  Yee Whye Teh,et al.  Generative Models as Distributions of Functions , 2021, ArXiv.

[45]  Carl E. Rasmussen,et al.  Gaussian Processes for Data-Efficient Learning in Robotics and Control , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Yoav Zemel,et al.  Procrustes Metrics on Covariance Operators and Optimal Transportation of Gaussian Processes , 2018, Sankhya A.

[47]  Yee Whye Teh,et al.  Attentive Neural Processes , 2019, ICLR.

[48]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[49]  Wenshuo Wang,et al.  Recurrent Attentive Neural Process for Sequential Data , 2019, ArXiv.

[50]  Eva L. Dyer,et al.  Hierarchical Optimal Transport for Multimodal Distribution Alignment , 2019, NeurIPS.

[51]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  R. Keener.  Theoretical Statistics: Topics for a Core Course , 2010 .

[53]  Anand Rangarajan,et al.  The Softassign Procrustes Matching Algorithm , 1997, IPMI.

[54]  Masayoshi Tomizuka,et al.  SERoCS: Safe and Efficient Robot Collaborative Systems for Next Generation Intelligent Industrial Co-Robots , 2018, ArXiv.