Scalable fine‐grained parallelization of plane‐wave–based ab initio molecular dynamics for large supercomputers

Many systems of great importance in material science, chemistry, solid‐state physics, and biophysics require forces generated from an electronic structure calculation, as opposed to an empirically derived force law to describe their properties adequately. The use of such forces as input to Newton's equations of motion forms the basis of the ab initio molecular dynamics method, which is able to treat the dynamics of chemical bond‐breaking and ‐forming events. However, a very large number of electronic structure calculations must be performed to compute an ab initio molecular dynamics trajectory, making the efficiency as well as the accuracy of the electronic structure representation critical issues. One efficient and accurate electronic structure method is the generalized gradient approximation to the Kohn–Sham density functional theory implemented using a plane‐wave basis set and atomic pseudopotentials. The marriage of the gradient‐corrected density functional approach with molecular dynamics, as pioneered by Car and Parrinello (R. Car and M. Parrinello, Phys Rev Lett 1985, 55, 2471), has been demonstrated to be capable of elucidating the atomic scale structure and dynamics underlying many complex systems at finite temperature. However, despite the relative efficiency of this approach, it has not been possible to obtain parallel scaling of the technique beyond several hundred processors on moderately sized systems using standard approaches. Consequently, the time scales that can be accessed and the degree of phase space sampling are severely limited. To take advantage of next generation computer platforms with thousands of processors such as IBM's BlueGene, a novel scalable parallelization strategy for Car–Parrinello molecular dynamics is developed using the concept of processor virtualization as embodied by the Charm++ parallel programming system. Charm++ allows the diverse elements of a Car–Parrinello molecular dynamics calculation to be interleaved with low latency such that unprecedented scaling is achieved. As a benchmark, a system of 32 water molecules, a common system size employed in the study of the aqueous solvation and chemistry of small molecules, is shown to scale on more than 1500 processors, which is impossible to achieve using standard approaches. This degree of parallel scaling is expected to open new opportunities for scientific inquiry. © 2004 Wiley Periodicals, Inc. J Comput Chem 16: 2006–2022, 2004

[1]  Mark R. Wilson,et al.  ‘Covalent’ effects in ‘ionic’ systems , 1996 .

[2]  M. Parrinello,et al.  The nature of the hydrated excess proton in water , 1999, Nature.

[3]  Roberto Car,et al.  Introduction to density-functional theory and ab-initio molecular dynamics , 2002 .

[4]  A. Becke,et al.  Density-functional exchange-energy approximation with correct asymptotic behavior. , 1988, Physical review. A, General physics.

[5]  M. Parrinello,et al.  Ab-Initio Molecular Dynamics: Principles and Practical Implementation , 1991 .

[6]  P. Fowler,et al.  Rotational spectra and structures of van der Waals dimers of Ar with a series of fluorocarbons: Ar⋅⋅⋅CH2CHF, Ar⋅⋅⋅CH2CF2, and Ar⋅⋅⋅CHFCF2 , 1991 .

[7]  M. Tuckerman.  Ab initio molecular dynamics: basic concepts, current trends and novel applications , 2002 .

[8]  J. D. Bernal,et al.  A Theory of Water and Ionic Solution, with Particular Reference to Hydrogen and Hydroxyl Ions , 1933 .

[9]  Paul Tavan,et al.  A hybrid method for solutes in complex solvents: Density functional theory combined with empirical force fields , 1999 .

[10]  Steven G. Johnson,et al.  The Fastest Fourier Transform in the West , 1997 .

[11]  Michele Parrinello,et al.  From silicon to RNA: The coming of age of ab initio molecular dynamics , 1997 .

[12]  M. Gillan.  The virtual matter laboratory , 1997 .

[13]  C. Dellago,et al.  Autoionization in Liquid Water , 2001, Science.

[14]  M. Parrinello,et al.  Superionic and metallic states of water and ammonia at giant planet conditions. , 1999, Science.

[15]  D. R. Hamann,et al.  Pseudopotentials that work: From H to Pu , 1982 .

[16]  Michele Parrinello,et al.  On the Quantum Nature of the Shared Proton in Hydrogen Bonds , 1997, Science.

[17]  Michiel Sprik,et al.  COMPUTER-SIMULATION OF THE DYNAMICS OF INDUCED POLARIZATION FLUCTUATIONS IN WATER , 1991 .

[18]  A. Zunger,et al.  Self-interaction correction to density-functional approximations for many-electron systems , 1981 .

[19]  Laxmikant V. Kalé,et al.  Supporting dynamic parallel object arrays , 2003, Concurr. Comput. Pract. Exp..

[20]  Ajay K. Royyuru,et al.  Blue Gene: A vision for protein science using a petaflop supercomputer , 2001, IBM Syst. J..

[21]  Mark E. Tuckerman,et al.  A reciprocal space based method for treating long range interactions in ab initio and force-field-based calculations in clusters , 1999 .

[22]  J. Perram,et al.  Simulation of electrostatic systems in periodic boundary conditions. I. Lattice sums and dielectric constants , 1980, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[23]  Michele Parrinello,et al.  Water Molecule Dipole in the Gas and in the Liquid Phase , 1999 .

[24]  Peter D. Haynes,et al.  Parallel fast Fourier transforms for electronic structure calculations , 2000 .

[25]  J. W. Humberston.  Classical mechanics , 1980, Nature.

[26]  R. Dreizler,et al.  Density-Functional Theory , 1990 .

[27]  R. Parr.  Density-functional theory of atoms and molecules , 1989 .

[28]  M. Klein,et al.  Quantum simulation studies of metal-ammonia solutions , 1994 .

[29]  Arieh Warshel,et al.  An empirical valence bond approach for comparing reactions in solutions and in enzymes , 1980 .

[30]  Aneesur Rahman,et al.  Correlations in the Motion of Atoms in Liquid Argon , 1964 .

[31]  Michele Parrinello,et al.  ELECTRONIC STRUCTURE OF THE CU, ZN SUPEROXIDE DISMUTASE ACTIVE SITE AND ITS INTERACTIONS WITH THE SUBSTRATE , 1995 .

[32]  Dawn A. Yarne,et al.  A dual length scale method for plane-wave-based, simulation studies of chemical systems modeled using mixed ab initio/empirical force field descriptions , 2001 .

[33]  E. M.,et al.  Statistical Mechanics , 2021, Manual for Theoretical Chemistry.

[34]  J. Ilja Siepmann,et al.  Thermodynamic Properties of the Williams, OPLS-AA, and MMFF94 All-Atom Force Fields for Normal Alkanes , 1998 .

[35]  Leonard Kleinman,et al.  Efficacious Form for Model Pseudopotentials , 1982 .

[36]  Mark E. Tuckerman,et al.  Ab Initio Molecular Dynamics Simulations , 1996 .

[37]  Mark E. Tuckerman,et al.  Exploiting multiple levels of parallelism in Molecular Dynamics based calculations via modern techniques and software paradigms on distributed memory computers , 2000 .

[38]  Laxmikant V. Kalé,et al.  A framework for collective personalized communication , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[39]  R. Car,et al.  A microscopic model for surface-induced diamond-to-graphite transitions , 1996, Nature.

[40]  M. Parrinello,et al.  Integrating the Car–Parrinello equations. III. Techniques for ultrasoft pseudopotentials , 1995 .

[41]  Laxmikant V. Kalé,et al.  Multiparadigm, Multilingual Interoperability: Experience with Converse , 1998, IPPS/SPDP Workshops.

[42]  Michele Parrinello,et al.  Integrating the Car–Parrinello equations. II. Multiple time scale techniques , 1994 .

[43]  R. Ashley,et al.  Fast pulsations in Wolf–Rayet stars , 1994, Nature.

[44]  Laxmikant V. Kalé,et al.  CHARM++: a portable concurrent object oriented system based on C++ , 1993, OOPSLA '93.

[45]  Kiyoyuki Terakura,et al.  First Principles Molecular Dynamics Study of Ziegler−Natta Heterogeneous Catalysis , 1998 .

[46]  M. Parrinello,et al.  Melting of Diamond at High Pressure , 1990, Science.

[47]  R. Car,et al.  First solvation shell of the Cu(II) aqua ion: evidence for fivefold coordination. , 2001, Science.

[48]  C. Catlow,et al.  Molecular dynamics studies of hydrocarbon diffusion in zeolites , 1991 .

[49]  Mark E. Tuckerman,et al.  Quantum dynamics via adiabatic ab initio centroid molecular dynamics , 1999 .

[50]  White,et al.  Implementation of gradient-corrected exchange-correlation potentials in Car-Parrinello total-energy calculations. , 1994, Physical review. B, Condensed matter.

[51]  Alexander D. MacKerell,et al.  All-atom empirical potential for molecular modeling and dynamics studies of proteins. , 1998, The journal of physical chemistry. B.

[52]  R. Car,et al.  Microscopic Growth Mechanisms for Carbon Nanotubes , 1997, Science.

[53]  Structure and dynamics of bipolarons in liquid ammonia. , 1992, Physical review letters.

[54]  D. Remler,et al.  Molecular dynamics without effective potentials via the Car-Parrinello approach , 1990 .

[55]  Johannes Grotendorst,et al.  Modern methods and algorithms of quantum chemistry , 2000 .

[56]  Frank H. Stillinger,et al.  Dynamics and ensemble averages for the polarization models of molecular interactions , 1979 .

[57]  Car,et al.  Unified approach for molecular dynamics and density-functional theory. , 1985, Physical review letters.

[58]  Mark E. Tuckerman,et al.  A new reciprocal space based treatment of long range interactions on surfaces , 2002 .

[59]  M. Parrinello,et al.  The nature and transport mechanism of hydrated hydroxide ions in aqueous solution , 2002, Nature.

[60]  W. Kohn,et al.  Self-Consistent Equations Including Exchange and Correlation Effects , 1965 .

[61]  M. Gillan,et al.  The melting curve of iron at the pressures of the Earth's core from ab initio calculations , 1999, Nature.

[62]  Tai-Sung Lee,et al.  A pseudobond approach to combining quantum mechanical and molecular mechanical methods , 1999 .

[63]  Parr,et al.  Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density. , 1988, Physical review. B, Condensed matter.

[64]  P. Kollman,et al.  A Second Generation Force Field for the Simulation of Proteins, Nucleic Acids, and Organic Molecules , 1995 .

[65]  M. Levitt,et al.  Theoretical studies of enzymic reactions: dielectric, electrostatic and steric stabilization of the carbonium ion in the reaction of lysozyme. , 1976, Journal of molecular biology.

[66]  Denis Caromel,et al.  Computing in Object-Oriented Parallel Environments , 2002, Lecture Notes in Computer Science.