Dynamics-Adapted Cone Kernels

We present a family of kernels for analysis of data generated by dynamical systems. These so-called cone kernels feature an explicit dependence on the dynamical vector field operating in the phase-space manifold, estimated empirically through finite-differences of time-ordered data samples. In particular, cone kernels assign strong affinity to pairs of samples whose relative displacement vector lies within a narrow cone aligned with the dynamical vector field. As a result, in a suitable asymptotic limit, the associated diffusion operator generates diffusions along the dynamical flow, and is invariant under a weakly restrictive class of transformations of the data, which includes conformal transformations. Moreover, the corresponding Dirichlet form is governed by the directional derivative of functions along the dynamical vector field. The latter feature is metric-independent. The diffusion eigenfunctions obtained via cone kernels are therefore adapted to the dynamics in that they vary predominantly in directions transverse to the flow. We demonstrate the utility of cone kernels in nonlinear flows on the 2-torus and North Pacific sea surface temperature data generated by a comprehensive climate model.

[1]  A. Majda,et al.  Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability , 2012, Proceedings of the National Academy of Sciences.

[2]  R. Vautard,et al.  Singular spectrum analysis in nonlinear dynamics, with applications to paleoclimatic time series , 1989 .

[3]  Kevin E. Trenberth,et al.  The Definition of El Niño. , 1997 .

[4]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[5]  Mathematics of Climate Modeling , 1997 .

[6]  P. Bérard,et al.  Embedding Riemannian manifolds by their heat kernel , 1994 .

[7]  N. Mantua,et al.  The Pacific Decadal Oscillation , 2002 .

[8]  Nadine Aubry,et al.  Preserving Symmetries in the Proper Orthogonal Decomposition , 1993, SIAM J. Sci. Comput..

[9]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[10]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[11]  Ronen Talmon,et al.  Empirical intrinsic geometry for nonlinear modeling and time series filtering , 2013, Proceedings of the National Academy of Sciences.

[12]  R. Coifman,et al.  Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions , 2006 .

[13]  Andrew J. Majda,et al.  Comparing low‐frequency and intermittent variability in comprehensive climate models through nonlinear Laplacian spectral analysis , 2012 .

[14]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[15]  Nadine Aubry,et al.  Spatiotemporal analysis of complex signals: Theory and applications , 1991 .

[16]  W. Collins,et al.  The Community Climate System Model Version 3 (CCSM3) , 2006 .

[17]  A. Majda,et al.  Data-driven methods for dynamical systems : Quantifying predictability and extracting spatiotemporal patterns , 2013 .

[18]  Andrew J. Majda,et al.  Time Series Reconstruction via Machine Learning: Revealing Decadal Variability and Intermittency in the North Pacific Sector of a Coupled Climate Model. , 2011, CIDU 2011.

[19]  V. Araújo.  Random Dynamical Systems , 2006, math/0608162.

[20]  R. A. Antonia,et al.  THE PHENOMENOLOGY OF SMALL-SCALE TURBULENCE , 1997 .

[21]  Yiying Tong,et al.  Discrete differential forms for computational modeling , 2005, SIGGRAPH Courses.

[22]  Michael Ghil,et al.  ADVANCED SPECTRAL METHODS FOR CLIMATIC TIME SERIES , 2002 .

[23]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[24]  K. Elworthy,et al.  The Geometry of Filtering , 2008, 0810.2253.

[25]  Vladimir Rokhlin,et al.  Randomized approximate nearest neighbors algorithm , 2011, Proceedings of the National Academy of Sciences.

[26]  C. Deser,et al.  The Reemergence of SST Anomalies in the North Pacific Ocean , 1999 .

[27]  Yakov Pesin,et al.  The Multiplicative Ergodic Theorem , 2013 .

[28]  James C. McWilliams,et al.  North Pacific Gyre Oscillation links ocean climate and ecosystem change , 2008 .

[29]  Andrew J. Majda,et al.  Strategies for Model Reduction: Comparing Different Optimal Bases , 2004 .

[30]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[31]  Andrew J. Majda,et al.  Limits of predictability in the North Pacific sector of a comprehensive climate model , 2012 .

[32]  G. P. King,et al.  Extracting qualitative dynamics from experimental data , 1986 .

[33]  Amit Singer,et al.  Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps , 2009, Proceedings of the National Academy of Sciences.

[34]  Andrew J. Majda,et al.  Intermittency, metastability and coarse graining for coupled deterministic–stochastic lattice systems , 2006 .

[35]  James P. Crutchfield,et al.  Geometry from a Time Series , 1980 .

[36]  Charles A. Micchelli,et al.  On Learning Vector-Valued Functions , 2005, Neural Computation.

[37]  Timothy D. Sauer,et al.  Time-Scale Separation from Diffusion-Mapped Delay Coordinates , 2013, SIAM J. Appl. Dyn. Syst..

[38]  A. Singer.  From graph to manifold Laplacian: The convergence rate , 2006 .

[39]  Markos A. Katsoulakis,et al.  Coarse-grained stochastic processes and kinetic Monte Carlo simulators for the diffusion of interacting particles , 2003 .

[40]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[41]  J. Portegies.  Embeddings of Riemannian Manifolds with Heat Kernels and Eigenfunctions , 2013, 1311.7568.

[42]  Ling Huang,et al.  An Analysis of the Convergence of Graph Laplacians , 2010, ICML.

[43]  Andrew J. Majda,et al.  Reemergence Mechanisms for North Pacific Sea Ice Revealed through Nonlinear Laplacian Spectral Analysis , 2014 .

[44]  Randall J. LeVeque,et al.  Finite difference methods for ordinary and partial differential equations - steady-state and time-dependent problems , 2007 .

[45]  P. R. Julian,et al.  Description of Global-Scale Circulation Cells in the Tropics with a 40–50 Day Period , 1972 .

[46]  Andrew J. Majda,et al.  Nonlinear Dynamics and Statistical Theories for Basic Geophysical Flows , 2006 .

[47]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[48]  M. Maggioni,et al.  Manifold parametrizations by eigenfunctions of the Laplacian and heat kernels , 2008, Proceedings of the National Academy of Sciences.

[49]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[50]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[51]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[52]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[53]  T. Berry,et al.  Model Free Techniques for Reduction of High-Dimensional Dynamics , 2013 .

[54]  Charles A. Micchelli,et al.  Universal Multi-Task Kernels , 2008, J. Mach. Learn. Res..

[55]  J. Zukas.  Introduction to the Modern Theory of Dynamical Systems , 1998 .

[56]  Andrew J. Majda,et al.  Nonlinear Laplacian spectral analysis: capturing intermittent and low‐frequency spatiotemporal patterns in high‐dimensional data , 2012, Stat. Anal. Data Min..

[57]  Dengyong Zhou,et al.  High-Order Regularization on Graphs , 2008 .

[58]  Michael Dellnitz,et al.  Computation of Essential Molecular Dynamics by Subdivision Techniques , 1996, Computational Molecular Dynamics.

[59]  S. Rosenberg.  The Laplacian on a Riemannian Manifold: The Construction of the Heat Kernel , 1997 .