Automatic Differentiable Monte Carlo: Theory and Application

Differentiable programming has emerged as a key programming paradigm empowering rapid developments of deep learning while its applications to important computational methods such as Monte Carlo remain largely unexplored. Here we present the general theory enabling infinite-order automatic differentiation on expectations computed by Monte Carlo with unnormalized probability distributions, which we call "automatic differentiable Monte Carlo" (ADMC). By implementing ADMC algorithms on computational graphs, one can also leverage state-of-the-art machine learning frameworks and techniques to traditional Monte Carlo applications in statistics and physics. We illustrate the versatility of ADMC by showing some applications: fast search of phase transitions and accurately finding ground states of interacting many-body models in two dimensions. ADMC paves a promising way to innovate Monte Carlo in various aspects to achieve higher accuracy and efficiency, e.g. easing or solving the sign problem of quantum many-body models through ADMC.

[1]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[2]  Juan Carrasquilla,et al.  Machine learning quantum phases of matter beyond the fermion sign problem , 2016, Scientific Reports.

[3]  L. Dixon,et al.  Automatic differentiation of algorithms , 2000 .

[4]  S. Sorella.  Generalized Lanczos algorithm for variational quantum Monte Carlo , 2000, cond-mat/0009149.

[5]  H. Yao,et al.  Sign-Problem-Free Fermionic Quantum Monte Carlo: Developments and Applications , 2018, Annual Review of Condensed Matter Physics.

[6]  Jimmy Ba,et al.  Kronecker-factored Curvature Approximations for Recurrent Neural Networks , 2018, ICLR.

[7]  R. Sugar,et al.  Monte Carlo calculations of coupled boson-fermion systems. I , 1981 .

[8]  F. Verstraete,et al.  Restricted Boltzmann Machines for Quantum States with Non-Abelian or Anyonic Symmetries. , 2019, Physical review letters.

[9]  Pieter Abbeel,et al.  Gradient Estimation Using Stochastic Computation Graphs , 2015, NIPS.

[10]  Andrew S. Darmawan,et al.  Restricted Boltzmann machine learning for solving strongly correlated quantum systems , 2017, 1709.06475.

[11]  S. Sorella.  GREEN FUNCTION MONTE CARLO WITH STOCHASTIC RECONFIGURATION , 1998, cond-mat/9803107.

[12]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[13]  Matthias Troyer,et al.  Computational complexity and fundamental limitations to fermionic quantum Monte Carlo simulations , 2004, Physical review letters.

[14]  B. Clark,et al.  Mitigating the Sign Problem through Basis Rotations. , 2019, Physical review letters.

[15]  A. Sandvik.  Finite-size scaling of the ground-state parameters of the two-dimensional Heisenberg model , 1997, cond-mat/9707123.

[16]  D. Ceperley,et al.  Monte Carlo simulation of a many-fermion study , 1977 .

[17]  A sufficient condition for the absence of the sign problem in the fermionic quantum Monte-Carlo algorithm , 2004, cond-mat/0407272.

[18]  Lei Wang,et al.  Split Orthogonal Group: A Guiding Principle for Sign-Problem-Free Fermionic Simulations. , 2015, Physical review letters.

[19]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  F. Y. Wu,et al.  The Ground State of Liquid He4 , 1962 .

[22]  Michael Figurnov,et al.  Monte Carlo Gradient Estimation in Machine Learning , 2019, J. Mach. Learn. Res..

[23]  Ruslan Salakhutdinov,et al.  Scaling up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix , 2015, ICML.

[24]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[25]  Emilie Huffman,et al.  Solution to sign problems in half-filled spin-polarized electronic systems , 2013, 1311.0034.

[26]  Chae-Yeun Park,et al.  On the geometry of learning neural quantum states , 2019 .

[27]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[28]  H. Yao,et al.  Solving the fermion sign problem in quantum Monte Carlo simulations by Majorana representation , 2014, 1408.2269.

[29]  Lei Wang,et al.  Differentiable Programming Tensor Networks , 2019, Physical Review X.

[30]  Ingo Roth,et al.  Easing the Monte Carlo sign problem , 2019, Science Advances.

[31]  Ankit B. Patel,et al.  Deep learning-enhanced variational Monte Carlo method for quantum many-body physics , 2019, Physical Review Research.

[32]  Roger G. Melko,et al.  Wave-function positivization via automatic differentiation , 2019, 1906.04654.

[33]  M. Troyer,et al.  Continuous-time Monte Carlo methods for quantum impurity models , 2010, 1012.4474.

[34]  White,et al.  Sign problem in the numerical simulation of many-electron systems. , 1990, Physical review. B, Condensed matter.

[35]  Dong-Ling Deng,et al.  Machine Learning Topological States , 2016, 1609.09060.

[36]  G. Carleo,et al.  Symmetries and Many-Body Excitations with Neural-Network Quantum States. , 2018, Physical review letters.

[37]  Matthias Troyer,et al.  Quantum Monte Carlo , 2004 .

[38]  Roger B. Grosse,et al.  Optimizing Neural Networks with Kronecker-factored Approximate Curvature , 2015, ICML.

[39]  Shimon Whiteson,et al.  DiCE: The Infinitely Differentiable Monte-Carlo Estimator , 2018, ICML.

[40]  J. Ignacio Cirac,et al.  Ground-state properties of quantum many-body systems: entangled-plaquette states and variational Monte Carlo , 2009, 0905.3898.

[41]  Reuven Y. Rubinstein,et al.  Sensitivity analysis of discrete event systems by the “push out” method , 1992, Ann. Oper. Res..

[42]  Huan He,et al.  Multi-Layer Restricted Boltzmann Machine Representation of 1D Quantum Many-Body Wave Functions , 2019, 1910.13454.

[43]  T. Xiang,et al.  Majorana Positivity and the Fermion Sign Problem of Quantum Monte Carlo Simulations. , 2016, Physical review letters.

[44]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[45]  Razvan Pascanu,et al.  Revisiting Natural Gradient for Deep Networks , 2013, ICLR.

[46]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[48]  Raphael Kaubruegger,et al.  Chiral topological phases from artificial neural networks , 2017, 1710.04713.

[49]  G. Carleo,et al.  Ground state phase diagram of the one-dimensional Bose-Hubbard model from restricted Boltzmann machines , 2019, Journal of Physics: Conference Series.

[50]  E. Berg,et al.  Sign-Problem–Free Quantum Monte Carlo of the Onset of Antiferromagnetism in Metals , 2012, Science.

[51]  C. Hubig.  Use and implementation of autodifferentiation in tensor network methods with complex scalars , 2019, 1907.13422.

[52]  Shimon Whiteson,et al.  A Baseline for Any Order Gradient Estimation in Stochastic Computation Graphs , 2019, ICML.

[53]  James Martens,et al.  New Insights and Perspectives on the Natural Gradient Method , 2014, J. Mach. Learn. Res..

[54]  R. Nieminen,et al.  Stochastic gradient approximation: An efficient method to optimize many-body wave functions , 1997 .

[55]  Matthias Troyer,et al.  Neural-network Quantum States , 2018 .

[56]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[57]  B. Clark,et al.  Variational optimization in the AI era: Computational Graph States and Supervised Wave-function Optimization , 2018, 1811.12423.

[58]  Jack P. C. Kleijnen,et al.  Optimization and Sensitivity Analysis of Computer Simulation Models by the Score Function Method , 1996 .

[59]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[60]  Zhou-Quan Wan,et al.  Automatic Differentiation for Complex Valued SVD , 2019, ArXiv.

[61]  Ira L. Karp.  Ground State of Liquid Helium , 1959 .

[62]  Satoshi Matsuoka,et al.  Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[64]  Roger B. Grosse,et al.  Distributed Second-Order Optimization using Kronecker-Factored Approximations , 2016, ICLR.

[65]  Charles C. Margossian,et al.  A review of automatic differentiation and its efficient implementation , 2018, WIREs Data Mining Knowl. Discov..

[66]  A. Sandvik.  Stochastic method for analytic continuation of quantum Monte Carlo data , 1998 .

[67]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[68]  Frank Noé,et al.  Deep-neural-network solution of the electronic Schrödinger equation , 2020, Nature Chemistry.

[69]  Ling Wang,et al.  Monte Carlo simulation with tensor network states , 2010, 1010.5450.

[70]  U. Schollwoeck.  The density-matrix renormalization group in the age of matrix product states , 2010, 1008.3477.

[71]  Yong-Sheng Zhang,et al.  Solving frustrated quantum many-particle models with convolutional neural networks , 2018, Physical Review B.

[72]  Hong Yao,et al.  Majorana-Time-Reversal Symmetries: A Fundamental Principle for Sign-Problem-Free Quantum Monte Carlo Simulations. , 2016, Physical review letters.

[73]  J. Cirac,et al.  Algorithms for finite projected entangled pair states , 2014, 1405.3259.

[74]  White,et al.  Density matrix formulation for quantum renormalization groups. , 1992, Physical review letters.

[75]  H. Saito.  Solving the Bose–Hubbard Model with Machine Learning , 2017, 1707.09723.

[76]  Yusuke Nomura,et al.  Constructing exact representations of quantum many-body systems with deep neural networks , 2018, Nature Communications.

[77]  Roger B. Grosse,et al.  A Kronecker-factored approximate Fisher matrix for convolution layers , 2016, ICML.

[78]  Jinguo Liu,et al.  Approximating quantum many-body wave functions using artificial neural networks , 2017, 1704.05148.