A Deep Learning Approach to Identify Local Structures in Atomic‐Resolution Transmission Electron Microscopy Images

Recording atomic-resolution transmission electron microscopy (TEM) images is becoming increasingly routine. A new bottleneck is then analyzing this information, which often involves time-consuming manual structural identification. We have developed a deep learning-based algorithm for recognition of the local structure in TEM images, which is stable to microscope parameters and noise. The neural network is trained entirely from simulation but is capable of making reliable predictions on experimental images. We apply the method to single sheets of defected graphene, and to metallic nanoparticles on an oxide support.

[1]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Antonio-José Almeida,et al.  NAT , 2019, Springer Reference Medizin.

[3]  L. Schultz,et al.  Atomic surface diffusion on Pt nanoparticles quantified by high-resolution transmission electron microscopy. , 2014, Micron.

[4]  J. Wagner,et al.  Accuracy of surface strain measurements from transmission electron microscopy images of nanoparticles , 2017, Advanced Structural and Chemical Imaging.

[5]  M. Haruta,et al.  Systematic morphology changes of gold nanoparticles supported on CeO2 during CO oxidation. , 2011, Angewandte Chemie.

[6]  J. Fuhrmann.  Advanced Computing In Electron Microscopy , 2016 .

[7]  Xianlong Wei,et al.  Direct Observation of the Layer-by-Layer Growth of ZnO Nanopillar by In situ High Resolution Transmission Electron Microscopy , 2017, Scientific Reports.

[8]  C. Koch.  Determination of core structure periodicity and point defect density along dislocations , 2002 .

[9]  R. Henderson,et al.  Comparison of optimal performance at 300 keV of three direct electron detectors for use in low dose electron microscopy , 2014, Ultramicroscopy.

[10]  Edward R. Dougherty,et al.  Mathematical Morphology in Image Processing , 1992 .

[11]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[12]  Ib Chorkendorff,et al.  Understanding the electrocatalysis of oxygen reduction on platinum and its alloys , 2012 .

[13]  Andrés Yáñez,et al.  The Peak Pairs algorithm for strain mapping from HRTEM images. , 2007, Ultramicroscopy.

[14]  Adv , 2019, International Journal of Pediatrics and Adolescent Medicine.

[15]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[16]  D. Van dyck,et al.  3D reconstruction of nanocrystalline particles from a single projection. , 2015, Micron.

[17]  Rama Vasudevan,et al.  Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations. , 2017, ACS nano.

[18]  G. Seifert,et al.  Electron knock-on cross section of carbon and boron nitride nanotubes , 2007 .

[19]  Jacob S. Vestergaard,et al.  Pattern recognition approach to quantify the atomic structure of graphene , 2014 .

[20]  Yanan Zhu,et al.  A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy , 2016, BMC Bioinformatics.

[21]  Meyer,et al.  Reconstruction of off‐axis electron holograms using a neural net , 1998 .

[22]  Martin Hÿtch,et al.  Quantitative measurement of displacement and strain fields from HREM micrographs , 1998 .

[23]  J. Schiøtz,et al.  Understanding the catalytic activity of gold nanoparticles through multi-scale simulations , 2011 .

[24]  Hogler Kirschner,et al.  Reasoning on domain knowledge level in human-computer interaction , 1994 .

[25]  P. Nellist,et al.  Rapid estimation of catalyst nanoparticle morphology and atomic-coordination by high-resolution Z-contrast electron microscopy. , 2014, Nano letters.

[26]  Xiaofeng Xu,et al.  In Situ Atom Scale Visualization of Domain Wall Dynamics in VO2 Insulator-Metal Phase Transition , 2014, Scientific Reports.

[27]  U. G. Vej-Hansen,et al.  Tuning the activity of Pt alloy electrocatalysts by means of the lanthanide contraction , 2016, Science.

[28]  S. Van Aert,et al.  Atom-counting in High Resolution Electron Microscopy:TEM or STEM - That's the question. , 2017, Ultramicroscopy.

[29]  P. Goodman,et al.  Numerical evaluations of N‐beam wave functions in electron scattering by the multi‐slice method , 1974 .

[30]  Jacob S. Vestergaard,et al.  Structure Identification in High-Resolution Transmission Electron Microscopic Images: An Example on Graphene , 2014, Microscopy and Microanalysis.

[31]  J. Nørskov,et al.  Ammonia Synthesis from First-Principles Calculations , 2005, Science.

[32]  M. Haruta,et al.  Visualizing Gas Molecules Interacting with Supported Nanoparticulate Catalysts at Reaction Conditions , 2012, Science.

[33]  Filippo Cavalca,et al.  Exploring the environmental transmission electron microscope. , 2012, Micron.

[34]  O. Brandt,et al.  Direct measurement of local lattice distortions in strained layer structures by HREM , 1993 .

[35]  M F Crommie,et al.  Direct imaging of lattice atoms and topological defects in graphene membranes. , 2008, Nano letters.

[36]  Christoph Hofer,et al.  Automated Image Acquisition for Low-Dose STEM at Atomic Resolution , 2017, Microscopy and Microanalysis.

[37]  Jacob Madsen.  Quantitative Image Simulation and Analysis of Nanoparticles , 2017 .

[38]  Michael Walter,et al.  The atomic simulation environment-a Python library for working with atoms. , 2017, Journal of physics. Condensed matter : an Institute of Physics journal.

[39]  Chem. , 2020, Catalysis from A to Z.

[40]  Jannik C. Meyer,et al.  Experimental analysis of charge redistribution due to chemical bonding by high-resolution transmission electron microscopy. , 2011, Nature materials.

[41]  J. Meyer,et al.  Progress in structure recovery from low dose exposures: Mixed molecular adsorption, exploitation of symmetry and reconstruction from the minimum signal level. , 2016, Ultramicroscopy.

[42]  K. Nagao,et al.  Experimental Observation of Quasicrystal Growth. , 2015, Physical Review Letters.

[43]  Haiyan Wang,et al.  Interface lattice displacement measurement to 1 pm by geometric phase analysis on aberration-corrected HAADF STEM images , 2013 .

[44]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[45]  Renu Sharma,et al.  Determination of atomic positions from time resolved high resolution transmission electron microscopy images. , 2018, Ultramicroscopy.

[46]  J. Biskupek,et al.  Electron dose dependence of signal-to-noise ratio, atom contrast and resolution in transmission electron microscope images. , 2014, Ultramicroscopy.

[47]  Hiroshi Sano,et al.  Novel Gold Catalysts for the Oxidation of Carbon Monoxide at a Temperature far Below 0 °C , 1987 .

[48]  Jian-Min Zuo,et al.  Lattice and strain analysis of atomic resolution Z-contrast images based on template matching. , 2014, Ultramicroscopy.

[49]  P. Nellist,et al.  Three-dimensional atomic models from a single projection using Z-contrast imaging: verification by electron tomography and opportunities. , 2017, Nanoscale.