Learning Stochastic Behaviour of Aggregate Data

Learning nonlinear dynamics of aggregate data is a challenging problem since the full trajectory of each individual is not observable, namely, the individual observed at one time point may not be observed at next time point. One class of existing work investigate such dynamics by requiring complete longitudinal individual-level trajectories. However, in most of the practical applications, the requirement is unrealistic due to technical limitations, experimental costs and/or privacy issues. The other one class of methods learn the dynamics by regarding aggregate behaviour as a stochastic process with/without hidden variable. The performances of such methods may be restricted due to complex dynamics, high dimensions and computation costs. In this paper, we propose a new weak form based framework to study the hidden dynamics of aggregate data via Wasserstein generative adversarial network(WGAN) and Fokker Planck Equation(FPE). Our model fall into the second class of methods with simple structure and computation. We demonstrate our approach in the context of a series of synthetic and real-world datasets.

[1]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[2]  Dilin Wang,et al.  Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm , 2016, NIPS.

[3]  Tommi S. Jaakkola,et al.  Learning Population-Level Diffusions with Generative RNNs , 2016, ICML.

[4]  Andrew J. Majda,et al.  Low-Dimensional Reduced-Order Models for Statistical Response and Uncertainty Quantification: Two-Layer Baroclinic Turbulence , 2016 .

[5]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[6]  S. Eddy.  Hidden Markov models. , 1996, Current opinion in structural biology.

[7]  S. Sharma,et al.  The Fokker-Planck Equation , 2010 .

[8]  Pat Langley,et al.  Crafting Papers on Machine Learning , 2000, ICML.

[9]  Michele Pavon,et al.  The Data‐Driven Schrödinger Bridge , 2018, Communications on Pure and Applied Mathematics.

[10]  Gang Bao,et al.  Weak Adversarial Networks for High-dimensional Partial Differential Equations , 2019, J. Comput. Phys..

[11]  John Langford,et al.  Learning nonlinear dynamic models , 2009, ICML '09.

[12]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter , 1990 .

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  M. V. Tretyakov,et al.  Stochastic Numerics for Mathematical Physics , 2004, Scientific Computation.

[15]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[16]  T. Nieuwenhuizen.  What are quantum fluctuations , 2007 .

[17]  Sebastian Becker,et al.  Solving stochastic differential equations and Kolmogorov equations by means of deep learning , 2018, ArXiv.

[18]  Allon M. Klein,et al.  Droplet Barcoding for Single-Cell Transcriptomics Applied to Embryonic Stem Cells , 2015, Cell.

[19]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[20]  Mingxiang Teng,et al.  On the widespread and critical impact of systematic bias and batch effects in single-cell RNA-Seq data , 2015 .

[21]  James Bailey,et al.  Learning Deep Hidden Nonlinear Dynamics from Aggregate Data , 2018, UAI 2018.

[22]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[23]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[24]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[25]  Geoffrey J. Gordon,et al.  Supervised Learning for Dynamical System Learning , 2015, NIPS.