Unsupervised segmentation of irradiation-induced order-disorder phase transitions in electron microscopy

We present a method for the unsupervised segmentation of electron microscopy images, which are powerful descriptors of materials and chemical systems. Images are oversegmented into overlapping chips, and similarity graphs are generated from embeddings extracted from a domain$\unicode{x2010}$pretrained convolutional neural network (CNN). The Louvain method for community detection is then applied to perform segmentation. The graph representation provides an intuitive way of presenting the relationship between chips and communities. We demonstrate our method to track irradiation$\unicode{x2010}$induced amorphous fronts in thin films used for catalysis and electronics. This method has potential for"on$\unicode{x2010}$the$\unicode{x2010}$fly"segmentation to guide emerging automated electron microscopes.

[1]  M. Scott,et al.  Generalization Across Experimental Parameters in Machine Learning Analysis of High Resolution Transmission Electron Microscopy Datasets , 2023, ArXiv.

[2]  F. Lin,et al.  A small-dataset-trained deep learning framework for identifying atoms on transmission electron microscopy images , 2023, Scientific Reports.

[3]  Yonina C. Eldar,et al.  DeepCut: Unsupervised Segmentation using Graph Neural Networks Clustering , 2022, 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[4]  R. Brutchey,et al.  Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis. , 2022, Nanoscale.

[5]  Le Wang,et al.  Resolving Diverse Oxygen Transport Pathways Across Sr‐Doped Lanthanum Ferrite and Metal‐Perovskite Heterostructures , 2022, Advanced Materials Interfaces.

[6]  J. Stuckner,et al.  Microstructure segmentation with deep learning encoders pre-trained on a large microscopy dataset , 2022, npj Computational Materials.

[7]  M. Ziatdinov,et al.  Deep learning for electron and scanning probe microscopy: From materials design to atomic fabrication , 2022, MRS Bulletin.

[8]  Xing Wu,et al.  Review in situ transmission electron microscope with machine learning , 2022, Journal of Semiconductors.

[9]  Kwanpyo Kim,et al.  STEM image analysis based on deep learning: identification of vacancy defects and polymorphs of MoS2 , 2022, Nano letters.

[10]  A. Vedaldi,et al.  Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Xiner Lu,et al.  In situ transmission electron microscopy and artificial intelligence enabled data analytics for energy materials , 2021, Journal of Energy Chemistry.

[12]  M. Olszta,et al.  Rapid and flexible segmentation of electron microscopy data using few-shot machine learning , 2021, npj Computational Materials.

[13]  Michael C. Cao,et al.  Uncovering material deformations via machine learning combined with four-dimensional scanning transmission electron microscopy , 2021, npj Computational Materials.

[14]  Steven R. Spurgeon,et al.  An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics , 2021, Microscopy and Microanalysis.

[15]  C. Freysoldt,et al.  Segmentation of Static and Dynamic Atomic-Resolution Microscopy Data Sets with Unsupervised Machine Learning Using Local Symmetry Descriptors , 2021, Microscopy and Microanalysis.

[16]  Sergei V. Kalinin,et al.  Experimental discovery of structure–property relationships in ferroelectric materials via active learning , 2021, Nature Machine Intelligence.

[17]  Sergei V. Kalinin,et al.  Towards automating structural discovery in scanning transmission electron microscopy , 2021, Microscopy and Microanalysis.

[18]  Colin Ophus,et al.  Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase-Contrast Transmission Electron Microscopy Images , 2020, Microscopy and Microanalysis.

[19]  Sergei V. Kalinin,et al.  Towards data-driven next-generation transmission electron microscopy , 2020, Nature Materials.

[20]  Yonggang Chen,et al.  An unsupervised image segmentation method combining graph clustering and high-level feature representation , 2020, Neurocomputing.

[21]  A. Alivisatos,et al.  AutoDetect-mNP: An Unsupervised Machine Learning Algorithm for Automated Analysis of Transmission Electron Microscope Images of Metal Nanoparticles , 2020, JACS Au.

[22]  S. Spurgeon Order-disorder behavior at thin film oxide interfaces , 2020, 2007.15136.

[23]  N. Pryds,et al.  Oxygen vacancies: The (in)visible friend of oxide electronics , 2020, Applied Physics Letters.

[24]  Catherine K. Groschner,et al.  Machine Learning Pipeline for Segmentation and Defect Identification from High-Resolution Transmission Electron Microscopy Data , 2020, Microscopy and Microanalysis.

[25]  S. Pennycook,et al.  A machine perspective of atomic defects in scanning transmission electron microscopy , 2019, InfoMat.

[26]  S. Bishop,et al.  Point Defects in Oxides: Tailoring Materials Through Defect Engineering , 2011 .

[27]  R. Hemley,et al.  Materials for Extreme Environments , 2009, Polymers and Additives in Extreme Environments.

[28]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[29]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[30]  Hartmann,et al.  Radiation tolerance of complex oxides , 2000, Science.

[31]  Feng Li,et al.  DefectNet: Toward Fast and Effective Defect Detection , 2021, IEEE Transactions on Instrumentation and Measurement.

[32]  Miss H. P. Narkhede A Review on Graph Based Segmentation , 2013 .