Shift‐and‐invert parallel spectral transformation eigensolver: Massively parallel performance for density‐functional based tight‐binding

The Shift‐and‐invert parallel spectral transformations (SIPs), a computational approach to solve sparse eigenvalue problems, is developed for massively parallel architectures with exceptional parallel scalability and robustness. The capabilities of SIPs are demonstrated by diagonalization of density‐functional based tight‐binding (DFTB) Hamiltonian and overlap matrices for single‐wall metallic carbon nanotubes, diamond nanowires, and bulk diamond crystals. The largest (smallest) example studied is a 128,000 (2000) atom nanotube for which ∼330,000 (∼5600) eigenvalues and eigenfunctions are obtained in ∼190 (∼5) seconds when parallelized over 266,144 (16,384) Blue Gene/Q cores. Weak scaling and strong scaling of SIPs are analyzed and the performance of SIPs is compared with other novel methods. Different matrix ordering methods are investigated to reduce the cost of the factorization step, which dominates the time‐to‐solution at the strong scaling limit. A parallel implementation of assembling the density matrix from the distributed eigenvectors is demonstrated. © 2015 Wiley Periodicals, Inc.

[1]  Walter Thiel,et al.  Semiempirical quantum–chemical methods , 2014 .

[2]  Chao Yang,et al.  Parallel eigenvalue calculation based on multiple shift-invert Lanczos and contour integral based spectral projection method , 2014, Parallel Comput..

[3]  Patrick Amestoy,et al.  Hybrid scheduling for the parallel solution of linear systems , 2006, Parallel Comput..

[4]  C. T. Kelley,et al.  Parallel implementation of electronic structure eigensolver using a partitioned folded spectrum method , 2015, 1502.07806.

[5]  Vipin Kumar,et al.  A Parallel Algorithm for Multilevel Graph Partitioning and Sparse Matrix Ordering , 1998, J. Parallel Distributed Comput..

[6]  William Gropp,et al.  Efficient Management of Parallelism in Object-Oriented Numerical Software Libraries , 1997, SciTools.

[7]  Axel Ruhe,et al.  The spectral transformation Lánczos method for the numerical solution of large sparse generalized symmetric eigenvalue problems , 1980 .

[8]  G. Seifert,et al.  Tight-binding density functional theory: an approximate Kohn-Sham DFT scheme. , 2007, The journal of physical chemistry. A.

[9]  Joost VandeVondele,et al.  Linear Scaling Self-Consistent Field Calculations with Millions of Atoms in the Condensed Phase. , 2012, Journal of chemical theory and computation.

[10]  Chao Yang,et al.  Accelerating atomic orbital-based electronic structure calculation via pole expansion and selected inversion , 2012, Journal of physics. Condensed matter : an Institute of Physics journal.

[11]  R. Bartlett,et al.  Coupled-cluster theory in quantum chemistry , 2007 .

[12]  James J. P. Stewart,et al.  Fast semiempirical calculations , 1982 .

[13]  François Pellegrini,et al.  PT-Scotch: A tool for efficient parallel graph ordering , 2008, Parallel Comput..

[14]  Lexing Ying,et al.  SelInv---An Algorithm for Selected Inversion of a Sparse Symmetric Matrix , 2011, TOMS.

[15]  J. Poulson.  Fast parallel solution of heterogeneous 3D time-harmonic wave equations , 2012 .

[16]  Gustavo E. Scuseria,et al.  What is the Best Alternative to Diagonalization of the Hamiltonian in Large Scale Semiempirical Calculations , 1999 .

[17]  Yousef Saad,et al.  A spectrum slicing method for the Kohn-Sham problem , 2012, Comput. Phys. Commun..

[18]  Charles H. Ward.  Materials Genome Initiative for Global Competitiveness , 2012 .

[19]  Yousef Saad,et al.  Numerical Methods for Electronic Structure Calculations of Materials , 2010, SIAM Rev..

[20]  P. Hohenberg,et al.  Inhomogeneous Electron Gas , 1964 .

[21]  Emanuel H. Rubensson,et al.  Kohn-Sham Density Functional Theory Electronic Structure Calculations with Linearly Scaling Computational Time and Memory Usage. , 2011, Journal of chemical theory and computation.

[22]  Stefan Grimme,et al.  n-Alkane isodesmic reaction energy errors in density functional theory are due to electron correlation effects. , 2010, Organic letters.

[23]  Sándor Suhai,et al.  Self-consistent-charge density-functional tight-binding method for simulations of complex materials properties , 1998 .

[24]  J. Sylvester.  XIX. A demonstration of the theorem that every homogeneous quadratic polynomial is reducible by real orthogonal substitutions to the form of a sum of positive and negative squares , 1852 .

[25]  Krishnan Raghavachari,et al.  Assessment of Gaussian-3 and Density Functional Theories for Enthalpies of Formation of C1−C16 Alkanes† , 2000 .

[26]  Chao Yang,et al.  Improving the scalability of a symmetric iterative eigensolver for multi‐core platforms , 2014, Concurr. Comput. Pract. Exp..

[27]  J. Pople,et al.  Approximate Self-Consistent Molecular Orbital Theory. I. Invariant Procedures , 1965 .

[28]  Robert A. van de Geijn,et al.  Elemental: A New Framework for Distributed Memory Dense Matrix Computations , 2013, TOMS.

[29]  Jiali Gao,et al.  Methods and Applications of Combined Quantum Mechanical and Molecular Mechanical Potentials , 2007 .

[30]  J. M. Martínez,et al.  Sparse Projected-Gradient Method As a Linear-Scaling Low-Memory Alternative to Diagonalization in Self-Consistent Field Electronic Structure Calculations. , 2013, Journal of chemical theory and computation.

[31]  E. Cuthill,et al.  Reducing the bandwidth of sparse symmetric matrices , 1969, ACM '69.

[32]  Gotthard Seifert,et al.  Density functional tight binding , 2014, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[33]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[34]  Foulkes,et al.  Tight-binding models and density-functional theory. , 1989, Physical review. B, Condensed matter.

[35]  Johannes Grotendorst,et al.  Performance of Dense Eigensolvers on BlueGene/Q , 2013, PPAM.

[36]  Michael C. Zerner,et al.  Triplet states via intermediate neglect of differential overlap: Benzene, pyridine and the diazines , 1976 .

[37]  Sergio Pissanetzky,et al.  Sparse Matrix Technology , 1984 .

[38]  S. Goedecker.  Linear scaling electronic structure methods , 1999 .

[39]  W. Kohn,et al.  Self-Consistent Equations Including Exchange and Correlation Effects , 1965 .

[40]  E Weinan,et al.  A Fast Parallel Algorithm for Selected Inversion of Structured Sparse Matrices with Application to 2D Electronic Structure Calculations , 2010, SIAM J. Sci. Comput..

[41]  Chao Yang,et al.  SIESTA-PEXSI: massively parallel method for efficient and accurate ab initio materials simulation without matrix diagonalization , 2014, Journal of physics. Condensed matter : an Institute of Physics journal.

[42]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM Rev..

[43]  J. Stewart.  Optimization of parameters for semiempirical methods I. Method , 1989 .

[44]  José E. Román,et al.  Strategies for spectrum slicing based on restarted Lanczos methods , 2012, Numerical Algorithms.

[45]  Nicolas Renon,et al.  A Sparse Self-Consistent Field Algorithm and Its Parallel Implementation: Application to Density-Functional-Based Tight Binding. , 2014, Journal of chemical theory and computation.

[46]  M. Yannakakis.  Computing the Minimum Fill-in is NP^Complete , 1981 .

[47]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[48]  Michael Sternberg,et al.  SIPs: Shift-and-invert parallel spectral transformations , 2007, TOMS.

[49]  K. Morokuma,et al.  ONIOM: A Multilayered Integrated MO + MM Method for Geometry Optimizations and Single Point Energy Predictions. A Test for Diels−Alder Reactions and Pt(P(t-Bu)3)2 + H2 Oxidative Addition , 1996 .

[50]  Vicente Hernández,et al.  SLEPc: A scalable and flexible toolkit for the solution of eigenvalue problems , 2005, TOMS.

[51]  D R Bowler,et al.  Calculations for millions of atoms with density functional theory: linear scaling shows its potential , 2009, Journal of physics. Condensed matter : an Institute of Physics journal.

[52]  A. George.  Nested Dissection of a Regular Finite Element Mesh , 1973 .

[53]  Ernst Schrem,et al.  Computer Implementation of the Finite-Element Procedure , 1973 .

[54]  James C. Sutherland,et al.  Graph-Based Software Design for Managing Complexity and Enabling Concurrency in Multiphysics PDE Software , 2011, TOMS.

[55]  M. Dewar,et al.  Ground States of Molecules. 38. The MNDO Method. Approximations and Parameters , 1977 .

[56]  Yang,et al.  Direct calculation of electron density in density-functional theory. , 1991, Physical review letters.

[57]  John R. Gilbert,et al.  Sparse Matrices in MATLAB: Design and Implementation , 1992, SIAM J. Matrix Anal. Appl..

[58]  J. C. Slater,et al.  Simplified LCAO Method for the Periodic Potential Problem , 1954 .

[59]  Chao Yang,et al.  New algorithms for iterative matrix‐free eigensolvers in quantum chemistry , 2015, J. Comput. Chem..

[60]  Emanuel H. Rubensson,et al.  Systematic sparse matrix error control for linear scaling electronic structure calculations , 2005, J. Comput. Chem..

[61]  Thom Vreven,et al.  Chapter 3 Hybrid Methods: ONIOM(QM:MM) and QM/MM , 2006 .

[62]  A Marek,et al.  The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and computational science , 2014, Journal of physics. Condensed matter : an Institute of Physics journal.

[63]  E Weinan,et al.  Pole-Based approximation of the Fermi-Dirac function , 2009, 0906.1319.

[64]  Paolo Bientinesi,et al.  High-Performance Solvers for Dense Hermitian Eigenproblems , 2012, SIAM J. Sci. Comput..

[65]  Patrick Amestoy,et al.  A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling , 2001, SIAM J. Matrix Anal. Appl..

[66]  T. Frauenheim,et al.  DFTB+, a sparse matrix-based implementation of the DFTB method. , 2007, The journal of physical chemistry. A.

[67]  Michael C. Zerner,et al.  Semiempirical Molecular Orbital Methods , 2007 .