A streaming multi-GPU implementation of image simulation algorithms for scanning transmission electron microscopy

Simulation of atomic-resolution image formation in scanning transmission electron microscopy can require significant computation times using traditional methods. A recently developed method, termed plane-wave reciprocal-space interpolated scattering matrix (PRISM), demonstrates potential for significant acceleration of such simulations with negligible loss of accuracy. Here, we present a software package called Prismatic for parallelized simulation of image formation in scanning transmission electron microscopy (STEM) using both the PRISM and multislice methods. By distributing the workload between multiple CUDA-enabled GPUs and multicore processors, accelerations as high as 1000 × for PRISM and 15 × for multislice are achieved relative to traditional multislice implementations using a single 4-GPU machine. We demonstrate a potentially important application of Prismatic, using it to compute images for atomic electron tomography at sufficient speeds to include in the reconstruction pipeline. Prismatic is freely available both as an open-source CUDA/C++ package with a graphical user interface and as a Python package, PyPrismatic.

[1]  Colin Ophus,et al.  Efficient linear phase contrast in scanning transmission electron microscopy with matched illumination and detector interferometry , 2016, Nature Communications.

[2]  Susanne Stemmer,et al.  Quantitative atomic resolution scanning transmission electron microscopy. , 2008, Physical review letters.

[3]  Wah Chiu,et al.  GENFIRE: A Generalized Fourier Iterative Reconstruction Algorithm for High-Resolution 3D Electron and X-ray Imaging , 2017, Microscopy and Microanalysis.

[4]  P. Stadelmann.  Image analysis and simulation software in transmission electron microscopy , 2003, Microscopy and Microanalysis.

[5]  Andres More,et al.  Intel Xeon Phi Coprocessor High Performance Programming , 2013 .

[6]  J. Zuo,et al.  Web-Based Electron Microscopy Application Software: Web-EMAPS , 2004, Microscopy and Microanalysis.

[7]  Akira Ohtomo,et al.  Atomic-scale imaging of nanoengineered oxygen vacancy profiles in SrTiO3 , 2004, Nature.

[8]  Pierre Stadelmann,et al.  EMS-A software package for electron diffraction analysis and HREM image simulation in materials science , 1987 .

[9]  J. M. Cowley,et al.  THE SCATTERING OF ELECTRONS BY ATOMS AND CRYSTALS. II. THE EFFECTS OF FINITE SOURCE SIZE , 1959 .

[10]  P. Nellist,et al.  Three-dimensional imaging in double aberration-corrected scanning confocal electron microscopy, part II: inelastic scattering. , 2008, Ultramicroscopy.

[11]  S. Pennycook.  The impact of STEM aberration correction on materials science. , 2017, Ultramicroscopy.

[12]  N. Shibata,et al.  Dynamics of annular bright field imaging in scanning transmission electron microscopy. , 2010, Ultramicroscopy.

[13]  W Van den Broek,et al.  FDES, a GPU-based multislice algorithm with increased efficiency of the computation of the projected potential. , 2015, Ultramicroscopy.

[14]  J. Miao,et al.  Atomic electron tomography: 3D structures without crystals , 2016, Science.

[15]  Benjamin Berkels,et al.  Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts , 2014, Nature Communications.

[16]  P. Nellist.  Scanning Transmission Electron Microscopy , 2020, Definitions.

[17]  J. Miao,et al.  Three-dimensional imaging of dislocations in a nanoparticle at atomic resolution , 2013, Nature.

[18]  V. Grillo,et al.  Accurate and Fast Multislice Simulations of HAADF Image Contrast by Parallel Computing , 2008 .

[19]  J. Verbeeck,et al.  Progress and new advances in simulating electron microscopy datasets using MULTEM. , 2016, Ultramicroscopy.

[20]  Y. Yao,et al.  STEM image simulation with hybrid CPU/GPU programming. , 2016, Ultramicroscopy.

[21]  C. Ophus.  A fast image simulation algorithm for scanning transmission electron microscopy , 2017, Advanced Structural and Chemical Imaging.

[22]  D Van Dyck,et al.  MULTEM: A new multislice program to perform accurate and fast electron diffraction and imaging simulations using Graphics Processing Units with CUDA. , 2015, Ultramicroscopy.

[23]  R. Bücker,et al.  Low-dose cryo electron ptychography via non-convex Bayesian optimization , 2017, Scientific Reports.

[24]  A. Rosenauer,et al.  STEMSIM—a New Software Tool for Simulation of STEM HAADF Z-Contrast Imaging , 2008 .

[25]  K. Ishizuka,et al.  A new theoretical and practical approach to the multislice method , 1977 .

[26]  A. Crewe,et al.  Scanning transmission electron microscopy * , 1974, Journal of microscopy.

[27]  Colin Ophus,et al.  Three-dimensional coordinates of individual atoms in materials revealed by electron tomography. , 2015, Nature materials.

[28]  P. Nellist,et al.  Three-dimensional imaging in double aberration-corrected scanning confocal electron microscopy, part I: elastic scattering. , 2008, Ultramicroscopy.

[29]  O. L. Krivanek,et al.  Sub-ångstrom resolution using aberration corrected electron optics , 2002, Nature.

[30]  Juri Barthel,et al.  Direct imaging of single Au atoms within GaAs nanowires. , 2012, Nano letters.

[31]  K. Ishizuka,et al.  A practical approach for STEM image simulation based on the FFT multislice method. , 2002, Ultramicroscopy.

[32]  J. M. Cowley,et al.  The scattering of electrons by atoms and crystals. I. A new theoretical approach , 1957 .

[33]  Z. Wang.  Introduction to Conventional Transmission Electron Microscopy , 2003 .

[34]  Andrew V. Martin,et al.  Quantum mechanical model for phonon excitation in electron diffraction and imaging using a Born-Oppenheimer approximation , 2010 .

[35]  T. Woehl,et al.  Dark-field image contrast in transmission scanning electron microscopy: Effects of substrate thickness and detector collection angle. , 2016, Ultramicroscopy.

[36]  Steven G. Johnson,et al.  The Design and Implementation of FFTW3 , 2005, Proceedings of the IEEE.

[37]  D. Muller.  Structure and bonding at the atomic scale by scanning transmission electron microscopy. , 2009, Nature materials.

[38]  Y. Wang,et al.  Origin of atomic displacement in HAADF image of the tilted specimen. , 2017, Ultramicroscopy.

[39]  V. Grillo,et al.  STEM_CELL: a software tool for electron microscopy: part 1--simulations. , 2013, Ultramicroscopy.

[40]  Kenneth W. Martin,et al.  Mastering CMake: A Cross-Platform Build System , 2008 .

[41]  K. Volz,et al.  STEMsalabim: A high-performance computing cluster friendly code for scanning transmission electron microscopy image simulations of thin specimens. , 2017, Ultramicroscopy.

[42]  Colin Ophus,et al.  Atomic scale imaging of competing polar states in a Ruddlesden–Popper layered oxide , 2016, Nature Communications.

[43]  C. Koch.  Determination of core structure periodicity and point defect density along dislocations , 2002 .

[44]  J. Miao,et al.  Deciphering chemical order/disorder and material properties at the single-atom level , 2016, Nature.

[45]  MALTS: A Tool to Simulate Lorentz Transmission Electron Microscopy From Micromagnetic Simulations , 2012, IEEE Transactions on Magnetics.

[46]  J. Miao,et al.  Electron tomography at 2.4-ångström resolution , 2012, Nature.

[47]  James Demmel,et al.  Benchmarking GPUs to tune dense linear algebra , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[48]  Wouter Van den Broek,et al.  Method for retrieval of the three-dimensional object potential by inversion of dynamical electron scattering. , 2012, Physical review letters.

[49]  J. Fuhrmann.  Advanced Computing In Electron Microscopy , 2016 .

[50]  Malcolm L. H. Green,et al.  Simultaneous atomic-resolution electron ptychography and Z-contrast imaging of light and heavy elements in complex nanostructures , 2016, Nature Communications.

[51]  Wah Chiu,et al.  GENFIRE: A generalized Fourier iterative reconstruction algorithm for high-resolution 3D imaging , 2017, Scientific Reports.

[52]  Wolfgang Dahmen,et al.  Poisson noise removal from high-resolution STEM images based on periodic block matching , 2015, Advanced Structural and Chemical Imaging.

[53]  M. Graef.  Introduction to Conventional Transmission Electron Microscopy: List of symbols , 2003 .

[54]  Jack J. Purdum,et al.  C programming guide , 1983 .

[55]  J. Silcox,et al.  Simulation of annular dark field stem images using a modified multislice method , 1987 .

[56]  P. Nellist,et al.  Unscrambling Mixed Elements using High Angle Annular Dark Field Scanning Transmission Electron Microscopy. , 2016, Physical review letters.

[57]  Marcelo Yuffe,et al.  The Implementation of the 65nm Dual-Core 64b Merom Processor , 2007, 2007 IEEE International Solid-State Circuits Conference. Digest of Technical Papers.