Design of a Graphical User Interface for Few-Shot Machine Learning Classification of Electron Microscopy Data

Electron Microscopy Data Christina Doty*, Shaun Gallagher*, Wenqi Cui*, Wenya Chen*, Shweta Bhushan*, Marjolein Oostrom, Sarah Akers, and Steven R. Spurgeon a) Department of Materials Science and Engineering, University of Washington, Seattle, Washington 98195 Department of Chemistry, University of Washington, Seattle, Washington 98195 Department of Electrical and Computer Engineering, University of Washington, Seattle, Washington 98195 Department of Chemical Engineering, University of Washington, Seattle, Washington 98195 National Security Directorate, Pacific Northwest National Laboratory, Richland, Washington 99352 Energy and Environment Directorate, Pacific Northwest National Laboratory, Richland, Washington 99352

[1]  J. A. Mir,et al.  Characterisation of the Medipix3 detector for 60 and 80keV electrons. , 2017, Ultramicroscopy.

[2]  Jeffrey M. Ede.  Deep learning in electron microscopy , 2020, Mach. Learn. Sci. Technol..

[3]  M. Olszta,et al.  Rapid and flexible segmentation of electron microscopy data using few-shot machine learning , 2021, npj Computational Materials.

[4]  Maxim Ziatdinov,et al.  Learning surface molecular structures via machine vision , 2017, npj Computational Materials.

[5]  S. Pennycook.  The impact of STEM aberration correction on materials science. , 2017, Ultramicroscopy.

[6]  A. Kirkland,et al.  Detectors—The ongoing revolution in scanning transmission electron microscopy and why this important to material characterization , 2020, APL Materials.

[7]  Arvind Satyanarayan,et al.  Vega-Lite: A Grammar of Interactive Graphics , 2018, IEEE Transactions on Visualization and Computer Graphics.

[8]  Furqan A. Shah,et al.  50 years of scanning electron microscopy of bone—a comprehensive overview of the important discoveries made and insights gained into bone material properties in health, disease, and taphonomy , 2019, Bone Research.

[9]  Artëm Yankov,et al.  Few-Shot Learning with Metric-Agnostic Conditional Embeddings , 2018, ArXiv.

[10]  R. Rai,et al.  Role of transmission electron microscopy in the semiconductor industry for process development and failure analysis , 2009 .

[11]  Isabella Haberbosch,et al.  Software tools for automated transmission electron microscopy , 2018, Nature Methods.

[12]  Angus I. Kirkland,et al.  Materials Advances through Aberration-Corrected Electron Microscopy , 2006 .

[13]  Q. Ramasse,et al.  Aberration-corrected scanning transmission electron microscopy for atomic-resolution studies of functional oxides , 2014 .

[14]  Zhan’ao Tan,et al.  Diverse applications of MoO3for high performance organic photovoltaics: fundamentals, processes and optimization strategies , 2020 .

[15]  S. Priya,et al.  Integration of SrTiO3 on crystallographically oriented epitaxial germanium for low-power device applications. , 2015, ACS applied materials & interfaces.

[16]  Paul M. Voyles,et al.  Informatics and data science in materials microscopy , 2017 .

[17]  Sergei V. Kalinin,et al.  Towards data-driven next-generation transmission electron microscopy , 2020, Nature Materials.

[18]  Sergei V. Kalinin,et al.  Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets , 2015, Advanced Structural and Chemical Imaging.

[19]  Tolga Tasdizen,et al.  Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning , 2019, Science Advances.

[20]  Sergei V. Kalinin,et al.  Lab on a beam—Big data and artificial intelligence in scanning transmission electron microscopy , 2019, MRS Bulletin.

[21]  Anthony J. Hickey,et al.  Reproducibility, sharing and progress in nanomaterial databases. , 2017, Nature nanotechnology.

[22]  Ondrej Dyck,et al.  Mapping mesoscopic phase evolution during E-beam induced transformations via deep learning of atomically resolved images , 2018, npj Computational Materials.

[23]  Fei Su,et al.  Deep learning analysis on microscopic imaging in materials science , 2020 .

[24]  Yingge Du,et al.  The effects of core-level broadening in determining band alignment at the epitaxial SrTiO3(001)/p-Ge(001) heterojunction , 2017 .

[25]  K. Jungjohann,et al.  Possibility of an integrated transmission electron microscope: enabling complex in-situ experiments , 2021, Journal of Materials Science.

[26]  Elizabeth Kautz,et al.  Rapid and Flexible Semantic Segmentation of Electron Microscopy Data Using Few-Shot Machine Learning , 2021 .

[27]  K. Pazdernik,et al.  Microstructural classification of unirradiated LiAlO2 pellets by deep learning methods , 2020 .

[28]  Thomas Boudier,et al.  EM-net: Deep learning for electron microscopy image segmentation , 2020 .

[29]  Anne L Plant,et al.  Improved reproducibility by assuring confidence in measurements in biomedical research , 2014, Nature Methods.

[30]  Sergei V. Kalinin,et al.  Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform , 2018, Advanced Structural and Chemical Imaging.

[31]  Jeffrey Heer,et al.  D³ Data-Driven Documents , 2011, IEEE Transactions on Visualization and Computer Graphics.

[32]  Seiji Takeda,et al.  Current status and future directions for in situ transmission electron microscopy. , 2016, Ultramicroscopy.

[33]  Maxim Ziatdinov,et al.  Phases and Interfaces from Real Space Atomically Resolved Data: Physics-Based Deep Data Image Analysis. , 2016, Nano letters.

[34]  Sergei V. Kalinin,et al.  Knowledge Extraction from Atomically Resolved Images. , 2017, ACS nano.

[35]  Jeffrey Heer,et al.  SpanningAspectRatioBank Easing FunctionS ArrayIn ColorIn Date Interpolator MatrixInterpola NumObjecPointI Rectang ISchedu Parallel Pause Scheduler Sequen Transition Transitioner Transiti Tween Co DelimGraphMLCon IData JSONCon DataField DataSc Dat DataSource Data DataUtil DirtySprite LineS RectSprite , 2011 .

[36]  Daniel C. Ralph,et al.  High Dynamic Range Pixel Array Detector for Scanning Transmission Electron Microscopy , 2016, Microscopy and Microanalysis.

[37]  D. Golberg,et al.  Recent Progress of In Situ Transmission Electron Microscopy for Energy Materials , 2019, Advanced materials.

[38]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[39]  Elizabeth A. Holm,et al.  High Throughput Quantitative Metallography for Complex Microstructures Using Deep Learning: A Case Study in Ultrahigh Carbon Steel , 2018, Microscopy and Microanalysis.

[40]  H. Sebastian Seung,et al.  Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification , 2017, Bioinform..

[41]  David T. Fullwood,et al.  Microstructure Sensitive Design for Performance Optimization , 2012 .

[42]  U. Krupp,et al.  Aachen-Heerlen annotated steel microstructure dataset , 2021, Scientific Data.

[43]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[44]  Elizabeth A. Holm,et al.  A computer vision approach for automated analysis and classification of microstructural image data , 2015 .

[45]  C. Hoermann,et al.  Hybrid pixel direct detector for electron energy loss spectroscopy. , 2020, Ultramicroscopy.

[46]  John Langford,et al.  Telling humans and computers apart automatically , 2004, CACM.

[47]  P. Withers,et al.  Rich multi-dimensional correlative imaging , 2019, IOP Conference Series: Materials Science and Engineering.