Fast and Robust Shortest Paths on Manifolds Learned from Data

We propose a fast, simple and robust algorithm for computing shortest paths and distances on Riemannian manifolds learned from data. This amounts to solving a system of ordinary differential equations (ODEs) subject to boundary conditions. Here standard solvers perform poorly because they require well-behaved Jacobians of the ODE, and usually, manifolds learned from data imply unstable and ill-conditioned Jacobians. Instead, we propose a fixed-point iteration scheme for solving the ODE that avoids Jacobians. This enhances the stability of the solver, while reduces the computational cost. In experiments involving both Riemannian metric learning and deep generative models we demonstrate significant improvements in speed and stability over both general-purpose state-of-the-art solvers as well as over specialized solvers.

[1]  Robert D. Russell,et al.  Numerical solution of boundary value problems for ordinary differential equations , 1995, Classics in applied mathematics.

[2]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[3]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[4]  J. Skilling.  Bayesian Solution of Ordinary Differential Equations , 1992 .

[5]  Michael R. Osborne,et al.  Numerical solution of boundary value problems for ordinary differential equations , 1995, Classics in applied mathematics.

[6]  Ben Calderhead,et al.  Probabilistic Linear Multistep Methods , 2016, NIPS.

[7]  Yuesheng Xu,et al.  Universal Kernels , 2006, J. Mach. Learn. Res..

[8]  Fixed points by mean value iterations , 1972 .

[9]  Ji-Huan He,et al.  Variational iteration method for autonomous ordinary differential systems , 2000, Appl. Math. Comput..

[10]  Harry van Zanten,et al.  Information Rates of Nonparametric Gaussian Process Methods , 2011, J. Mach. Learn. Res..

[11]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[12]  Simo Särkkä,et al.  A probabilistic model for the numerical solution of initial value problems , 2016, Statistics and Computing.

[13]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[14]  Neil D. Lawrence,et al.  Metrics for Probabilistic Geometries , 2014, UAI.

[15]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[16]  Lars Kai Hansen,et al.  Latent Space Oddity: on the Curvature of Deep Generative Models , 2017, ICLR.

[17]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[18]  P. Deuflhard.  Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms , 2011 .

[19]  Philipp Hennig,et al.  Probabilistic Line Searches for Stochastic Optimization , 2015, NIPS.

[20]  Søren Hauberg,et al.  Geodesic exponential kernels: When curvature and linearity conflict , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Mark A. Girolami,et al.  Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems , 2016, ArXiv.

[22]  M. Girolami,et al.  Bayesian Solution Uncertainty Quantification for Differential Equations , 2013 .

[23]  Søren Hauberg,et al.  A Geometric take on Metric Learning , 2012, NIPS.

[24]  A fixed point iterative method for the solution of two-point boundary value problems for a second order differential equations , 2017, Alexandria Engineering Journal.

[25]  Hossein Jafari,et al.  A comparison between the variational iteration method and the successive approximations method , 2014, Appl. Math. Lett..

[26]  David Duvenaud,et al.  Probabilistic ODE Solvers with Runge-Kutta Means , 2014, NIPS.

[27]  Michael A. Osborne,et al.  Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees , 2015, NIPS.

[28]  Donato Trigiante,et al.  A Hybrid Mesh Selection Strategy Based on Conditioning for Boundary Value ODE Problems , 2004, Numerical Algorithms.

[29]  W. R. Mann,et al.  Mean value methods in iteration , 1953 .

[30]  Philipp Hennig,et al.  Convergence rates of Gaussian ODE filters , 2018, Statistics and Computing.

[31]  Dino Sejdinovic,et al.  Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences , 2018, ArXiv.

[32]  Lars Kai Hansen,et al.  A Locally Adaptive Normal Distribution , 2016, NIPS.

[33]  É. Picard.  Mémoire sur la théorie des équations aux dérivées partielles et la méthode des approximations successives , 1890 .

[34]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[35]  Philipp Hennig,et al.  Active Uncertainty Calibration in Bayesian ODE Solvers , 2016, UAI.

[36]  Søren Hauberg,et al.  Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics , 2013, AISTATS.

[37]  Suheil A. Khuri,et al.  Variational iteration method: Green's functions and fixed point iterations perspective , 2014, Appl. Math. Lett..

[38]  Søren Hauberg,et al.  A Random Riemannian Metric for Probabilistic Shortest-Path Tractography , 2015, MICCAI.

[39]  S. Ishikawa.  Fixed points by a new iteration method , 1974 .

[40]  G. Wahba.  Spline models for observational data , 1990 .

[41]  Mark A. Girolami,et al.  Bayesian Quadrature for Multiple Related Integrals , 2018, ICML.