Energy landscapes for a machine learning application to series data.

Methods developed to explore and characterise potential energy landscapes are applied to the corresponding landscapes obtained from optimisation of a cost function in machine learning. We consider neural network predictions for the outcome of local geometry optimisation in a triatomic cluster, where four distinct local minima exist. The accuracy of the predictions is compared for fits using data from single and multiple points in the series of atomic configurations resulting from local geometry optimisation and for alternative neural networks. The machine learning solution landscapes are visualised using disconnectivity graphs, and signatures in the effective heat capacity are analysed in terms of distributions of local minima and their properties.

[1]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[2]  Vladimir A Mandelshtam,et al.  Self-consistent phonons revisited. I. The role of thermal versus quantum fluctuations on structural transitions in large Lennard-Jones clusters. , 2012, The Journal of chemical physics.

[3]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[4]  David J Wales,et al.  Energy landscapes for diffusion: analysis of cage-breaking processes. , 2008, The Journal of chemical physics.

[5]  H. Scheraga,et al.  Monte Carlo-minimization approach to the multiple-minima problem in protein folding. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Diego Prada-Gracia,et al.  Exploring the Free Energy Landscape: From Dynamics to Networks and Back , 2009, PLoS Comput. Biol..

[7]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  M. Karplus,et al.  The topology of multidimensional potential energy surfaces: Theory and application to peptide structure and kinetics , 1997 .

[9]  D. Goldfarb.  A family of variable-metric methods derived by variational means , 1970 .

[10]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[11]  David J Wales,et al.  Decoding the energy landscape: extracting structure, dynamics and thermodynamics , 2012, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[12]  Thomas A. Weber,et al.  Inherent pair correlation in simple liquids , 1984 .

[13]  Mark N. Gibbs,et al.  Combining ab initio computations, neural networks, and diffusion Monte Carlo: An efficient method to treat weakly bound molecules , 1996 .

[14]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[15]  Daan Frenkel,et al.  Visualizing basins of attraction for different minimization algorithms. , 2013, The journal of physical chemistry. B.

[16]  G. Henkelman,et al.  A dimer method for finding saddle points on high dimensional potential surfaces using only first derivatives , 1999 .

[17]  Edward Teller,et al.  Interaction of the van der Waals Type Between Three Atoms , 1943 .

[18]  Jonathan P. K. Doye,et al.  An order parameter approach to coexistence in atomic clusters , 1995 .

[19]  Musa A. Mammadov,et al.  From Convex to Nonconvex: A Loss Function Analysis for Binary Classification , 2010, 2010 IEEE International Conference on Data Mining Workshops.

[20]  Jonathan P. K. Doye,et al.  Coexistence and phase separation in clusters: From the small to the not‐so‐small regime , 1995 .

[21]  Janet E. Jones,et al.  On the Calculation of Certain Crystal Potential Constants, and on the Cubic Crystal of Least Potential Energy , 1925 .

[22]  David J Wales,et al.  Energy landscapes: some new horizons. , 2010, Current opinion in structural biology.

[23]  D. L. Freeman,et al.  Phase changes in 38-atom Lennard-Jones clusters. I. A parallel tempering study in the canonical ensemble , 2000, physics/0003068.

[24]  David J. Wales,et al.  Some further applications of discrete path sampling to cluster isomerization , 2004 .

[25]  Mark A. Miller,et al.  Archetypal energy landscapes , 1998, Nature.

[26]  Dimitrios Maroudas,et al.  Phase behavior of the 38-atom Lennard-Jones cluster. , 2014, The Journal of chemical physics.

[27]  F. Noé,et al.  Transition networks for modeling the kinetics of conformational change in macromolecules. , 2008, Current opinion in structural biology.

[28]  F. Stillinger,et al.  A Topographic View of Supercooled Liquids and Glass Formation , 1995, Science.

[29]  David J. Wales,et al.  Surveying a complex potential energy landscape: Overcoming broken ergodicity using basin-sampling , 2013 .

[30]  Vladimir A Mandelshtam,et al.  Solid-solid structural transformations in Lennard-Jones clusters: accurate simulations versus the harmonic superposition approximation. , 2007, The journal of physical chemistry. A.

[31]  David J Wales,et al.  Symmetrisation schemes for global optimisation of atomic clusters. , 2013, Physical chemistry chemical physics : PCCP.

[32]  Vladimir A Mandelshtam,et al.  Structural transitions and melting in LJ(74-78) Lennard-Jones clusters from adaptive exchange Monte Carlo simulations. , 2006, The journal of physical chemistry. A.

[33]  E. M. Kleinberg,et al.  Stochastic discrimination , 1990, Annals of Mathematics and Artificial Intelligence.

[34]  Jonathan P. K. Doye,et al.  Characterization of anharmonicities on complex potential energy surfaces: Perturbation theory and simulation , 2001 .

[35]  Lindsey J. Munro,et al.  DEFECT MIGRATION IN CRYSTALLINE SILICON , 1999 .

[36]  Pekka Koskinen,et al.  Structural relaxation made simple. , 2006, Physical review letters.

[37]  David J. Wales,et al.  Basins of attraction for stationary points on a potential-energy surface , 1992 .

[38]  J. Doye,et al.  Global Optimization by Basin-Hopping and the Lowest Energy Structures of Lennard-Jones Clusters Containing up to 110 Atoms , 1997, cond-mat/9803344.

[39]  Manuel Athènes,et al.  Simulating structural transitions by direct transition current sampling: the example of LJ38. , 2011, The Journal of chemical physics.

[40]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[41]  D. L. Freeman,et al.  Phase changes in 38-atom Lennard-Jones clusters. II. A parallel tempering study of equilibrium and dynamic properties in the molecular dynamics and microcanonical ensembles , 2000, physics/0003072.

[42]  Vladimir A Mandelshtam,et al.  Low-temperature structural transitions: circumventing the broken-ergodicity problem. , 2007, Physical review letters.

[43]  F. Calvo,et al.  Free-energy landscapes from adaptively biased methods: application to quantum systems. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[44]  D. Wales.  Locating stationary points for clusters in cartesian coordinates , 1993 .

[45]  David J. Wales,et al.  Free energy surfaces from an extended harmonic superposition approach and kinetics for alanine dipeptide , 2008 .

[46]  D. Wales.  Discrete path sampling , 2002 .

[47]  J. Doye,et al.  THE DOUBLE-FUNNEL ENERGY LANDSCAPE OF THE 38-ATOM LENNARD-JONES CLUSTER , 1998, cond-mat/9808265.

[48]  David J. Wales,et al.  Coexistence in small inert gas clusters , 1993 .

[49]  Daniel A. Braun,et al.  Thermodynamics as a theory of decision-making with information-processing costs , 2012, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[50]  D. Wales,et al.  A doubly nudged elastic band method for finding transition states. , 2004, The Journal of chemical physics.

[51]  Harold A. Scheraga,et al.  Structure and free energy of complex thermodynamic systems , 1988 .

[52]  Jonathan P. K. Doye,et al.  Calculation of thermodynamic properties of small Lennard‐Jones clusters incorporating anharmonicity , 1995 .