Data-driven discovery of interacting particle systems using Gaussian processes

Interacting particle or agent systems that display a rich variety of collection motions are ubiquitous in science and engineering. A fundamental and challenging goal is to understand the link between individual interaction rules and collective behaviors. In this paper, we study the data-driven discovery of distance-based interaction laws in second-order interacting particle systems. We propose a learning approach that models the latent interaction kernel functions as Gaussian processes, which can simultaneously fulfill two inference goals: one is the nonparametric inference of interaction kernel function with the pointwise uncertainty quantification, and the other one is the inference of unknown parameters in the non-collective forces of the system. We formulate learning interaction kernel functions as a statistical inverse problem and provide a detailed analysis of recoverability conditions, establishing that a coercivity condition is sufficient for recoverability. We provide a finite-sample analysis, showing that our posterior mean estimator converges at an optimal rate equal to the one in the classical 1-dimensional Kernel Ridge regression. Numerical results on systems that exhibit different collective behaviors demonstrate efficient learning of our approach from scarce noisy trajectory data.

[1]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[2]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[3]  Mauro Maggioni,et al.  Data-driven Discovery of Emergent Behaviors in Collective Dynamics , 2020, Physica D. Nonlinear phenomena.

[4]  G. Parisi,et al.  Interaction ruling animal collective behavior depends on topological rather than metric distance: Evidence from a field study , 2007, Proceedings of the National Academy of Sciences.

[5]  Neil D. Lawrence,et al.  Gaussian Processes for Big Data , 2013, UAI.

[6]  R. Fetecau,et al.  Emergent behaviour in multi-particle systems with non-local interactions , 2013 .

[7]  I. Couzin,et al.  Inferring the structure and dynamics of interactions in schooling fish , 2011, Proceedings of the National Academy of Sciences.

[8]  Fei Lu,et al.  Learning interaction kernels in heterogeneous systems of agents from multiple trajectories , 2019, J. Mach. Learn. Res..

[9]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[10]  S. Smale,et al.  Shannon sampling II: Connections to learning theory , 2005 .

[11]  Lorenzo Rosasco,et al.  Learning from Examples as an Inverse Problem , 2005, J. Mach. Learn. Res..

[12]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[13]  E. D. Vito,et al.  Fast Rates for Regularized Least-squares Algorithm , 2005 .

[14]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[15]  Houman Owhadi,et al.  Solving and Learning Nonlinear PDEs with Gaussian Processes , 2021, J. Comput. Phys..

[16]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[17]  Mauro Maggioni,et al.  Learning theory for inferring interaction kernels in second-order interacting agent systems , 2020, Sampling Theory, Signal Processing, and Data Analysis.

[18]  Ming Zhong,et al.  Nonparametric inference of interaction laws in systems of agents from trajectory data , 2018, Proceedings of the National Academy of Sciences.

[19]  Giacomo Albi,et al.  Stability Analysis of Flock and Mill Rings for Second Order Models in Swarming , 2013, SIAM J. Appl. Math..

[20]  Nicole Abaid,et al.  Fish in a ring: spatio-temporal pattern formation in one-dimensional animal groups , 2010, Journal of The Royal Society Interface.

[21]  Dino Sejdinovic,et al.  Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences , 2018, ArXiv.

[22]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[23]  A. Caponnetto,et al.  Optimal Rates for the Regularized Least-Squares Algorithm , 2007, Found. Comput. Math..

[24]  Mauro Maggioni,et al.  Learning Interaction Kernels in Stochastic Systems of Interacting Particles from Multiple Trajectories , 2020, Foundations of Computational Mathematics.

[25]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[26]  A. Tikhonov,et al.  Numerical Methods for the Solution of Ill-Posed Problems , 1995 .

[27]  Michaell Taylor,et al.  Towards a Mathematical Theory of Influence and Attitude Change , 1968 .

[28]  John N. Tsitsiklis,et al.  On Krause's Multi-Agent Consensus Model With State-Dependent Connectivity , 2008, IEEE Transactions on Automatic Control.

[29]  Pavel Exner,et al.  Hilbert Space Operators in Quantum Physics , 1994 .

[30]  Gilles Blanchard,et al.  Optimal Rates for Regularization of Statistical Inverse Learning Problems , 2016, Found. Comput. Math..

[31]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[32]  Lulu Kang,et al.  Gaussian Process Assisted Active Learning of Physical Laws , 2019, Technometrics.

[33]  M. Rudelson,et al.  Hanson-Wright inequality and sub-gaussian concentration , 2013 .

[34]  Xiang Zhou,et al.  EXPLICIT ESTIMATION OF DERIVATIVES FROM DATA AND DIFFERENTIAL EQUATIONS BY GAUSSIAN PROCESS REGRESSION , 2020, International Journal for Uncertainty Quantification.

[35]  Roland Hostettler,et al.  State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[37]  Sébastien Motsch,et al.  Heterophilious Dynamics Enhances Consensus , 2013, SIAM Rev..

[38]  Debdeep Pati,et al.  Frequentist coverage and sup-norm convergence rate in Gaussian process regression , 2017, 1708.04753.

[39]  I. Kevrekidis,et al.  Coarse-scale PDEs from fine-scale observations via machine learning , 2019, Chaos.

[40]  Markus Heinonen,et al.  LEARNING STOCHASTIC DIFFERENTIAL EQUATIONS WITH GAUSSIAN PROCESSES WITHOUT GRADIENT MATCHING , 2018, 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP).

[41]  Markus Heinonen,et al.  Learning unknown ODE models with Gaussian processes , 2018, ICML.

[42]  Dan Cornford,et al.  Gaussian Process Approximations of Stochastic Differential Equations , 2007, Gaussian Processes in Practice.

[43]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[44]  A. Bertozzi,et al.  State Transitions and the Continuum Limit for a 2D Interacting, Self-Propelled Particle System , 2006, nlin/0606031.

[45]  Leah Edelstein-Keshet,et al.  Inferring individual rules from collective behavior , 2010, Proceedings of the National Academy of Sciences.

[46]  Felipe Cucker,et al.  Emergent Behavior in Flocks , 2007, IEEE Transactions on Automatic Control.

[47]  Mathieu Lewin,et al.  The Crystallization Conjecture: A Review , 2015, 1504.01153.

[48]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[49]  Meng Li,et al.  Non-asymptotic Analysis in Kernel Ridge Regression , 2020, ArXiv.

[50]  U. Krause A DISCRETE NONLINEAR AND NON–AUTONOMOUS MODEL OF CONSENSUS FORMATION , 2007 .

[51]  D. Sumpter Collective Animal Behavior , 2010 .

[52]  A. Bertozzi,et al.  Self-propelled particles with soft-core interactions: patterns, stability, and collapse. , 2006, Physical review letters.

[53]  Zhiping Mao,et al.  Nonlocal Flocking Dynamics: Learning the Fractional Order of PDEs from Particle Simulations , 2018, Communications on Applied Mathematics and Computation.

[54]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.