Discovering phases, phase transitions, and crossovers through unsupervised machine learning: A critical examination.

We apply unsupervised machine learning techniques, mainly principal component analysis (PCA), to compare and contrast the phase behavior and phase transitions in several classical spin models-the square- and triangular-lattice Ising models, the Blume-Capel model, a highly degenerate biquadratic-exchange spin-1 Ising (BSI) model, and the two-dimensional XY model-and we examine critically what machine learning is teaching us. We find that quantified principal components from PCA not only allow the exploration of different phases and symmetry-breaking, but they can distinguish phase-transition types and locate critical points. We show that the corresponding weight vectors have a clear physical interpretation, which is particularly interesting in the frustrated models such as the triangular antiferromagnet, where they can point to incipient orders. Unlike the other well-studied models, the properties of the BSI model are less well known. Using both PCA and conventional Monte Carlo analysis, we demonstrate that the BSI model shows an absence of phase transition and macroscopic ground-state degeneracy. The failure to capture the "charge" correlations (vorticity) in the BSI model (XY model) from raw spin configurations points to some of the limitations of PCA. Finally, we employ a nonlinear unsupervised machine learning procedure, the "autoencoder method," and we demonstrate that it too can be trained to capture phase transitions and critical points.

[1]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[2]  John D. Dow,et al.  Zinc-blende—diamond order-disorder transition in metastable crystalline(GaAs)1−xGe2xalloys , 1983 .

[3]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[4]  Wooseop Kwak,et al.  First-order phase transition and tricritical scaling behavior of the Blume-Capel model: A Wang-Landau sampling approach. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  V. J. Emery,et al.  Ising Model for the ? Transition and Phase Separation in He^{3}-He^{4} Mixtures , 1971 .

[6]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[7]  Matthew M. Graham,et al.  Asymptotically exact conditional inference in deep generative models and differentiable simulators , 2016 .

[8]  Alexander G. Gray,et al.  Introduction to astroML: Machine learning for astrophysics , 2012, 2012 Conference on Intelligent Data Understanding.

[9]  J. Eisert,et al.  Area laws for the entanglement entropy - a review , 2008, 0808.3773.

[10]  M. Nielsen,et al.  Entanglement in a simple quantum phase transition , 2002, quant-ph/0202162.

[11]  Yang Qi,et al.  Self-learning Monte Carlo method , 2016, 1610.03137.

[12]  Richard Neuberg,et al.  Projected Regression Methods for Inverting Fredholm Integrals: Formalism and Application to Analytical Continuation , 2016 .

[13]  Ce-Jun Liu,et al.  Behavior of damage spreading in the two-dimensional Blume-Capel model. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  J. A. Plascak,et al.  Critical behavior of the spin- 3 2 Blume-Capel model in two dimensions , 1998 .

[15]  Giacomo Torlai,et al.  Neural Decoder for Topological Codes. , 2016, Physical review letters.

[16]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[17]  Matthias Troyer,et al.  Solving the quantum many-body problem with artificial neural networks , 2016, Science.

[18]  S. Huber,et al.  Learning phase transitions by confusion , 2016, Nature Physics.

[19]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[20]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[21]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[22]  J. Crutchfield Between order and chaos , 2011, Nature Physics.

[23]  A. Nihat Berker,et al.  Orderings of a stacked frustrated triangular system in three dimensions , 1984 .

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Lei Wang,et al.  Discovering phase transitions with unsupervised learning , 2016, 1606.00318.

[26]  J. Cardy,et al.  Entanglement entropy and quantum field theory , 2004, hep-th/0405152.

[27]  D. Landau,et al.  Monte Carlo study of the fcc Blume-Capel model , 1980 .

[28]  L. Onsager Crystal statistics. I. A two-dimensional model with an order-disorder transition , 1944 .

[29]  H. Capel On the possibility of first-order phase transitions in Ising systems of triplet ions with zero-field , 1966 .

[30]  David J. Schwab,et al.  An exact mapping between the Variational Renormalization Group and Deep Learning , 2014, ArXiv.

[31]  White,et al.  Density matrix formulation for quantum renormalization groups. , 1992, Physical review letters.

[32]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[33]  J. A. Plascak,et al.  Dynamics of rough surfaces generated by two-dimensional lattice spin models. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[34]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[35]  Richard A. Lewis,et al.  Drug design by machine learning: the use of inductive logic programming to model the structure-activity relationships of trimethoprim analogues binding to dihydrofolate reductase. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Roger G. Melko,et al.  Learning Thermodynamics with Boltzmann Machines , 2016, ArXiv.

[37]  J. Eisert,et al.  Colloquium: Area laws for the entanglement entropy , 2010 .

[38]  Graham D. Bruce,et al.  Single-atom imaging of fermions in a quantum-gas microscope , 2015, Nature Physics.

[39]  Marcos Rigol,et al.  Numerical linked-cluster algorithms. I. Spin systems on square, triangular, and kagomé lattices. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[40]  Immanuel Bloch,et al.  Single-atom-resolved fluorescence imaging of an atomic Mott insulator , 2010, Nature.

[41]  Olsson Monte Carlo analysis of the two-dimensional XY model. II. Comparison with the Kosterlitz renormalization-group equations. , 1995, Physical review. B, Condensed matter.

[42]  J. Cardy Scaling and Renormalization in Statistical Physics , 1996 .

[43]  M. Blume THEORY OF THE FIRST-ORDER MAGNETIC PHASE CHANGE IN UO$sub 2$ , 1966 .

[44]  Andrea J. Liu,et al.  A structural approach to relaxation in glassy liquids , 2015, Nature Physics.

[45]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[46]  Kyumin Lee,et al.  Uncovering social spammers: social honeypots + machine learning , 2010, SIGIR.

[47]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[48]  J. Stephenson,et al.  Ising‐Model Spin Correlations on the Triangular Lattice. III. Isotropic Antiferromagnetic Lattice , 1970 .

[49]  D. Deng,et al.  Quantum Entanglement in Neural Network States , 2017, 1701.04844.

[50]  J. Stephenson,et al.  Ising‐Model Spin Correlations on the Triangular Lattice , 1964 .

[51]  P G Wolynes,et al.  Learning To Fold Proteins Using Energy Landscape Theory. , 2013, Israel journal of chemistry.

[52]  G. Wannier,et al.  Antiferromagnetism. The Triangular Ising Net , 1950 .

[53]  R. Moessner,et al.  Interplay of quantum and thermal fluctuations in a frustrated magnet , 2003 .

[54]  Serena Bradde,et al.  PCA Meets RG , 2016, Journal of Statistical Physics.

[55]  Eytan Barouch,et al.  Statistical Mechanics of the X Y Model. II. Spin-Correlation Functions , 1971 .

[56]  Christian Van den Broeck,et al.  Statistical Mechanics of Learning , 2001 .

[57]  Juan Carrasquilla,et al.  Machine learning quantum phases of matter beyond the fermion sign problem , 2016, Scientific Reports.

[58]  RadhaKanta Mahapatra,et al.  Business data mining - a machine learning perspective , 2001, Inf. Manag..

[59]  D. Mukamel,et al.  Ising model for tricritical points in ternary mixtures , 1974 .

[60]  Sebastian Johann Wetzel,et al.  Unsupervised learning of phase transitions: from principal component analysis to variational autoencoders , 2017, Physical review. E.

[61]  Yves Chauvin,et al.  Backpropagation: theory, architectures, and applications , 1995 .

[62]  J. Kosterlitz,et al.  The critical properties of the two-dimensional xy model , 1974 .

[63]  R. Melko,et al.  Machine Learning Phases of Strongly Correlated Fermions , 2016, Physical Review X.

[64]  Jeff Z Y Chen,et al.  Identifying polymer states by machine learning. , 2017, Physical review. E.

[65]  Wolff,et al.  Collective Monte Carlo updating for spin systems. , 1989, Physical review letters.

[66]  Hamid Ez-Zahraouy,et al.  Phase diagrams of the spin-1 Blume-Capel film with an alternating crystal field , 2004 .

[67]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[68]  Lukasz Cincio,et al.  Entropy of entanglement and correlations induced by a quench: Dynamics of a quantum phase transition in the quantum Ising model , 2007 .

[69]  Rémi Bardenet,et al.  Monte Carlo Methods , 2013, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[70]  H. Katzgraber Introduction to Monte Carlo Methods , 2009, 0905.1629.

[71]  Austin Howard,et al.  Applications of Monte Carlo Methods to Statistical Physics , 2009 .

[72]  Joseph Lajzerowicz,et al.  Spin-1 lattice-gas model. I. Condensation and solidification of a simple fluid , 1975 .

[73]  R. T. Scalettar,et al.  Monte Carlo study of an inhomogeneous Blume-Capel model: A case study of the local density approximation , 2008 .

[74]  Kurt Binder,et al.  Phase diagrams and critical behavior in Ising square lattices with nearest- and next-nearest-neighbor interactions , 1980 .

[75]  David M. Reif,et al.  Machine Learning for Detecting Gene-Gene Interactions , 2006, Applied bioinformatics.

[76]  Markus Greiner,et al.  A quantum gas microscope for detecting single atoms in a Hubbard-regime optical lattice , 2009, Nature.

[77]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[78]  M. Blume Theory of the First‐Order Magnetic Phase Change in UO2 , 1966 .

[79]  Roger G. Melko,et al.  Machine learning phases of matter , 2016, Nature Physics.