Unsupervised learning of phase transitions: from principal component analysis to variational autoencoders

We examine unsupervised machine learning techniques to learn features that best describe configurations of the two-dimensional Ising model and the three-dimensional XY model. The methods range from principal component analysis over manifold and clustering methods to artificial neural-network-based variational autoencoders. They are applied to Monte Carlo-sampled configurations and have, a priori, no knowledge about the Hamiltonian or the order parameter. We find that the most promising algorithms are principal component analysis and variational autoencoders. Their predicted latent parameters correspond to the known order parameters. The latent representations of the models in question are clustered, which makes it possible to identify phases without prior knowledge of their existence. Furthermore, we find that the reconstruction loss function can be used as a universal identifier for phase transitions.

[1]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[2]  L. Onsager.  Crystal statistics. I. A two-dimensional model with an order-disorder transition , 1944 .

[3]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[4]  N. Mermin,et al.  Absence of Ferromagnetism or Antiferromagnetism in One- or Two-Dimensional Isotropic Heisenberg Models , 1966 .

[5]  L. Kadanoff.  Scaling laws for Ising models near T(c) , 1966 .

[6]  P. Hohenberg.  Existence of Long-Range Order in One and Two Dimensions , 1967 .

[7]  Philip W. Anderson,et al.  Resonating valence bonds: A new kind of insulator? , 1973 .

[8]  K. Wilson.  The renormalization group: Critical phenomena and the Kondo problem , 1975 .

[9]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[10]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[11]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[12]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[13]  A.P.Gottlob,et al.  Critical Behaviour of the 3D XY-Model: A Monte Carlo Study , 1993, cond-mat/9305020.

[14]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[15]  Kristin A. Persson,et al.  Predicting crystal structures with data mining of quantum calculations. , 2003, Physical review letters.

[16]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[17]  T. Seideman,et al.  Current-driven oscillations and time-dependent transport in nanojunctions. , 2005, Physical review letters.

[18]  A. Rollett,et al.  The Monte Carlo Method , 2004 .

[19]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[20]  S Jesse,et al.  Disorder identification in hysteresis data: recognition analysis of the random-bond-random-field Ising model. , 2009, Physical review letters.

[21]  Anubhav Jain,et al.  Finding Nature’s Missing Ternary Oxide Compounds Using Machine Learning and Density Functional Theory , 2010 .

[22]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[23]  Prabhat,et al.  Classification of nodal pockets in many-electron wave functions via machine learning , 2012, Journal of Mathematical Chemistry.

[24]  Tara N. Sainath,et al.  FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597 , 2012 .

[25]  Klaus-Robert Müller,et al.  Finding Density Functionals with Machine Learning , 2011, Physical review letters.

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Yousef Saad,et al.  Data mining for materials: Computational experiments with AB compounds , 2012 .

[28]  K. Müller,et al.  Fast and accurate modeling of molecular atomization energies with machine learning. , 2011, Physical review letters.

[29]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[30]  David J. Schwab,et al.  An exact mapping between the Variational Renormalization Group and Deep Learning , 2014, ArXiv.

[31]  O. A. von Lilienfeld,et al.  Machine learning for many-body physics: The case of the Anderson impurity model , 2014, 1408.1143.

[32]  Zhenwei Li,et al.  Molecular dynamics with on-the-fly machine learning of quantum-mechanical forces. , 2015, Physical review letters.

[33]  James E. Gubernatis,et al.  Structure classification and melting temperature prediction in octet AB solids via machine learning , 2015 .

[34]  Lei Wang,et al.  Discovering phase transitions with unsupervised learning , 2016, 1606.00318.

[35]  Roger G. Melko,et al.  Learning Thermodynamics with Boltzmann Machines , 2016, ArXiv.

[36]  Roger G. Melko,et al.  Machine learning phases of matter , 2016, Nature Physics.

[37]  S. Huber,et al.  Learning phase transitions by confusion , 2016, Nature Physics.

[38]  Matthias Troyer,et al.  Solving the quantum many-body problem with artificial neural networks , 2016, Science.

[39]  G. Kasieczka,et al.  Deep-learning top taggers or the end of QCD? , 2017, 1701.08784.