GPU algorithms for density matrix methods on MOPAC: linear scaling electronic structure calculations for large molecular systems

Purification of the density matrix methods should be employed when dealing with complex chemical systems containing many atoms. The running times for these methods scale linearly with the number of atoms if we consider the sparsity from the density matrix. Since the efficiency expected from those methods is closely tied to the underlying parallel implementations of the linear algebra operations (e.g., P2 = P × P), we proposed a central processing unit (CPU) and graphics processing unit (GPU) parallel matrix-matrix multiplication in SVBR (symmetrical variable block row) format for energy calculations through the SP2 algorithm. This algorithm was inserted in MOPAC’s MOZYME method, using the original LMO Fock matrix assembly, and the atomic integral calculation implemented on it. Correctness and performance tests show that the implemented SP2 is accurate and fast, as the GPU is able to achieve speedups up to 40 times for a water cluster system with 42,312 orbitals running in one NVIDIA K40 GPU card compared to the single-threaded version. The GPU-accelerated SP2 algorithm using the MOZYME LMO framework enables the calculations of semiempirical wavefunction with stricter SCF criteria for localized charged molecular systems, as well as the single-point energies of molecules with more than 100.000 LMO orbitals in less than 1 h. Graphical abstract Parallel CPU and GPU purification algorithms for electronic structure calculations were implemented in MOPAC’s MOZYME method. Some matrices in these calculations, e.g., electron density P, are compressed, and the developed linear algebra operations deal with non-zero entries only. We employed the NVIDIA/CUDA platform to develop GPU algorithms, and accelerations up to 40 times for larger systems were achieved. Parallel CPU and GPU purification algorithms for electronic structure calculations were implemented in MOPAC’s MOZYME method. Some matrices in these calculations, e.g., electron density P, are compressed, and the developed linear algebra operations deal with non-zero entries only. We employed the NVIDIA/CUDA platform to develop GPU algorithms, and accelerations up to 40 times for larger systems were achieved.

[1]  Joost VandeVondele,et al.  Linear Scaling Self-Consistent Field Calculations with Millions of Atoms in the Condensed Phase. , 2012, Journal of chemical theory and computation.

[2]  Frank Neese,et al.  The ORCA program system , 2012 .

[3]  Alessandro Curioni,et al.  Semiempirical Molecular Dynamics (SEMD) I: Midpoint-Based Parallel Sparse Matrix-Matrix Multiplication Algorithm for Matrices with Decay. , 2015, Journal of chemical theory and computation.

[4]  Anders M N Niklasson,et al.  Computing the Density Matrix in Electronic Structure Theory on Graphics Processing Units. , 2012, Journal of chemical theory and computation.

[5]  Eamonn F. Healy,et al.  Development and use of quantum mechanical molecular models. 76. AM1: a new general purpose quantum mechanical molecular model , 1985 .

[6]  Walter Thiel,et al.  A semiempirical model for the two-center repulsion integrals in the NDDO approximation , 1977 .

[7]  Joost VandeVondele,et al.  Sparse matrix multiplication: The distributed block-compressed sparse row library , 2014, Parallel Comput..

[8]  Thierry Deutsch,et al.  Challenges in large scale quantum mechanical calculations , 2016, 1609.00252.

[9]  Matt Challacombe,et al.  An Optimized Sparse Approximate Matrix Multiply for Matrices with Decay , 2012, SIAM J. Sci. Comput..

[10]  Alán Aspuru-Guzik,et al.  Advances in molecular quantum chemistry contained in the Q-Chem 4 program package , 2014, Molecular Physics.

[11]  Darrin M. York,et al.  Recent Advances toward a General Purpose Linear-Scaling Quantum Force Field , 2014, Accounts of chemical research.

[12]  Evgeny Epifanovsky,et al.  A General Sparse Tensor Framework for Electronic Structure Theory. , 2017, Journal of chemical theory and computation.

[13]  T.Miyazaki,et al.  Parallel Sparse Matrix Multiplication for Linear Scaling Electronic Structure Calculations , 2000 .

[14]  Christian Ochsenfeld,et al.  Locality and Sparsity of Ab Initio One-Particle Density Matrices and Localized Orbitals , 1998 .

[15]  Nicolas Renon,et al.  A Sparse Self-Consistent Field Algorithm and Its Parallel Implementation: Application to Density-Functional-Based Tight Binding. , 2014, Journal of chemical theory and computation.

[16]  James J. P. Stewart,et al.  Application of the PM6 method to modeling proteins , 2009, Journal of molecular modeling.

[17]  David C. Clary,et al.  Quantum Chemistry of Complex Systems , 2006, Science.

[18]  Sergio Pissanetzky,et al.  Sparse Matrix Technology , 1984 .

[19]  S. Goedecker.  Linear scaling electronic structure methods , 1999 .

[20]  V M Anisimov,et al.  LocalSCF method for semiempirical quantum-chemical calculation of ultralarge biomolecules. , 2004, The Journal of chemical physics.

[21]  Lin-Wang Wang,et al.  Divide-and-conquer quantum mechanical material simulations with exascale supercomputers , 2014 .

[22]  K Fukushima,et al.  An insight into the general relationship between the three dimensional structures of enzymes and their electronic wave functions: Implication for the prediction of functional sites of enzymes , 2008, Proteins.

[23]  Kazuo Kitaura,et al.  Exploring chemistry with the fragment molecular orbital method. , 2012, Physical chemistry chemical physics : PCCP.

[24]  Todd J. Martínez,et al.  Charge Transfer and Polarization in Solvated Proteins from Ab Initio Molecular Dynamics , 2011 .

[25]  Mark S. Johnson,et al.  Generating Conformer Ensembles Using a Multiobjective Genetic Algorithm , 2007, J. Chem. Inf. Model..

[26]  D R Bowler,et al.  Calculations for millions of atoms with density functional theory: linear scaling shows its potential , 2009, Journal of physics. Condensed matter : an Institute of Physics journal.

[27]  José Mario Martínez,et al.  Globally convergent trust-region methods for self-consistent field electronic structure calculations. , 2004, The Journal of chemical physics.

[28]  R. Mcweeny,et al.  The density matrix in self-consistent field theory I. Iterative construction of the density matrix , 1956, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[29]  James J. P. Stewart,et al.  Application of localized molecular orbitals to the solution of semiempirical self‐consistent field equations , 1996 .

[30]  Emanuel H. Rubensson,et al.  Ergo: An open-source program for linear-scaling electronic structure calculations , 2018, SoftwareX.

[31]  Robert Zaleśny,et al.  Linear-Scaling Techniques in Computational Chemistry and Physics , 2011 .

[32]  L. Martínez,et al.  Density-based Globally Convergent Trust-region Methods for Self-consistent Field Electronic Structure Calculations , 2006 .

[33]  S. Pissanetzky.  Sparse Matrix Algebra , 1984 .

[34]  Roland Lindh,et al.  Utilizing high performance computing for chemistry: parallel computational chemistry. , 2010, Physical chemistry chemical physics : PCCP.

[35]  V. Anisimov,et al.  Charge transfer effects in the GroEL-GroES chaperonin tetramer in solution. , 2012, The journal of physical chemistry. B.

[36]  J. M. Martínez,et al.  Sparse Projected-Gradient Method As a Linear-Scaling Low-Memory Alternative to Diagonalization in Self-Consistent Field Electronic Structure Calculations. , 2013, Journal of chemical theory and computation.

[37]  R. Mcweeny.  Some Recent Advances in Density Matrix Theory , 1960 .

[38]  Wallace D. Fragoso,et al.  Assessment of semiempirical enthalpy of formation in solution as an effective energy function to discriminate native‐like structures in protein decoy sets , 2016, J. Comput. Chem..

[39]  Yihan Shao,et al.  Sparse matrix multiplications for linear scaling electronic structure calculations in an atom‐centered basis set using multiatom blocks , 2003, J. Comput. Chem..

[40]  Gustavo E. Scuseria,et al.  What is the Best Alternative to Diagonalization of the Hamiltonian in Large Scale Semiempirical Calculations , 1999 .

[41]  Kenneth M Merz,et al.  Divide-and-Conquer Hartree-Fock Calculations on Proteins. , 2010, Journal of chemical theory and computation.

[42]  Emanuel H. Rubensson,et al.  Assessment of density matrix methods for linear scaling electronic structure calculations , 2011, Journal of physics. Condensed matter : an Institute of Physics journal.

[43]  David E. Manolopoulos,et al.  Canonical purification of the density matrix in electronic-structure theory , 1998 .

[44]  D. Bowler,et al.  O(N) methods in electronic structure calculations. , 2011, Reports on progress in physics. Physical Society.

[45]  Gustavo E. Scuseria,et al.  Semiempirical methods with conjugate gradient density matrix search to replace diagonalization for molecular systems containing thousands of atoms , 1997 .

[46]  Alfredo Mayall Simas,et al.  RM1: A reparameterization of AM1 for H, C, N, O, P, S, F, Cl, Br, and I , 2006, J. Comput. Chem..

[47]  D. Moses,et al.  Excitation spectrum for ultrafast photogeneration of charged solitons in polyacetylene , 2002 .

[48]  Anders M N Niklasson,et al.  Computation of the Density Matrix in Electronic Structure Theory in Parallel on Multiple Graphics Processing Units. , 2014, Journal of chemical theory and computation.

[49]  Kazuo Kitaura,et al.  The Fragment Molecular Orbital Method: Practical Applications to Large Molecular Systems , 2009 .

[50]  J. Stewart,et al.  Self-consistent field convergence for proteins: a comparison of full and localized-molecular-orbital schemes , 2014, Journal of Molecular Modeling.

[51]  Joost VandeVondele,et al.  cp2k: atomistic simulations of condensed matter systems , 2014 .

[52]  Kenneth M. Merz,et al.  Linear scaling molecular orbital calculations of biological systems using the semiempirical divide and conquer method , 2000, J. Comput. Chem..

[53]  Elias Rudberg,et al.  Difficulties in applying pure Kohn–Sham density functional theory electronic structure methods to protein molecules , 2011, Journal of physics. Condensed matter : an Institute of Physics journal.

[54]  Julio Daniel Carvalho Maia,et al.  GPU Linear Algebra Libraries and GPGPU Programming for Accelerating MOPAC Semiempirical Quantum Chemistry Calculations. , 2012, Journal of chemical theory and computation.

[55]  Heather J Kulik,et al.  Ab initio quantum chemistry for protein structures. , 2012, The journal of physical chemistry. B.

[56]  Frank Neese,et al.  Sparse maps—A systematic infrastructure for reduced-scaling electronic structure methods. I. An efficient and simple linear scaling local MP2 method that uses an intermediate basis of pair natural orbitals. , 2015, The Journal of chemical physics.

[57]  A. V. Mitin,et al.  The dynamic “level shift” method for improving the convergence of the SCF procedure , 1988 .

[58]  Anders M.N. Niklasson.  Expansion algorithm for the density matrix , 2002 .

[59]  Joost VandeVondele,et al.  Combining Linear-Scaling DFT with Subsystem DFT in Born-Oppenheimer and Ehrenfest Molecular Dynamics Simulations: From Molecules to a Virus in Solution. , 2016, Journal of chemical theory and computation.

[60]  Mark S. Johnson,et al.  Accurate conformation‐dependent molecular electrostatic potentials for high‐throughput in silico drug discovery , 2009, J. Comput. Chem..

[61]  James J. P. Stewart,et al.  Fast semiempirical calculations , 1982 .