Self-supervised learning and prediction of microstructure evolution with convolutional recurrent neural networks

Summary Microstructural evolution is a key aspect of understanding and exploiting the processing-structure-property relationship of materials. Modeling microstructure evolution usually relies on coarse-grained simulations with evolution principles described by partial differential equations (PDEs). Here we demonstrate that convolutional recurrent neural networks can learn the underlying physical rules and replace PDE-based simulations in the prediction of microstructure phenomena. Neural nets are trained by self-supervised learning with image sequences from simulations of several common processes, including plane-wave propagation, grain growth, spinodal decomposition, and dendritic crystal growth. The trained networks can accurately predict both short-term local dynamics and long-term statistical properties of microstructures assessed herein and are capable of extrapolating beyond the training datasets in spatiotemporal domains and configurational and parametric spaces. Such a data-driven approach offers significant advantages over PDE-based simulations in time-stepping efficiency and offers a useful alternative, especially when the material parameters or governing PDEs are not well determined.

[1]  M. Harmer,et al.  Materials informatics for the screening of multi-principal elements and high-entropy alloys , 2019, Nature Communications.

[2]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[3]  J. E. Hilliard,et al.  Free Energy of a Nonuniform System. I. Interfacial Free Energy and Free Energy of a Nonuniform System. III. Nucleation in a Two‐Component Incompressible Fluid , 2013 .

[4]  Zhiliang Xu,et al.  Neural Time-Dependent Partial Differential Equation , 2020, ArXiv.

[5]  Chiho Kim,et al.  Machine Learning Assisted Predictions of Intrinsic Dielectric Breakdown Strength of ABX3 Perovskites , 2016 .

[6]  I. Steinbach,et al.  Topological phase inversion after long-term thermal exposure of nickel-base superalloys: Experiment and phase-field simulation , 2017 .

[7]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[8]  R. Batra,et al.  Physically informed artificial neural networks for atomistic modeling of materials , 2018, Nature Communications.

[9]  Erin Antono,et al.  Building Data-driven Models with Microstructural Images: Generalization and Interpretability , 2017, ArXiv.

[10]  Long-Qing Chen.  Phase-Field Models for Microstructure Evolution , 2002 .

[11]  Franz Aurenhammer,et al.  Voronoi Diagrams , 2000, Handbook of Computational Geometry.

[12]  P. Alam.  ‘W’ , 2021, Composites Engineering.

[13]  P. Alam.  ‘T’ , 2021, Composites Engineering: An A–Z Guide.

[14]  Bülent Yener,et al.  Image driven machine learning methods for microstructure recognition , 2016 .

[15]  Felix A Faber,et al.  Machine Learning Energies of 2 Million Elpasolite (ABC_{2}D_{6}) Crystals. , 2015, Physical review letters.

[16]  Stefano Curtarolo,et al.  Finding Unprecedentedly Low-Thermal-Conductivity Half-Heusler Semiconductors via High-Throughput Materials Modeling , 2014, 1401.2439.

[17]  Nian Liu,et al.  Visualizing Battery Reactions and Processes by Using In Situ and In Operando Microscopies , 2018 .

[18]  Jake Graser,et al.  Machine Learning and Energy Minimization Approaches for Crystal Structure Predictions: A Review and New Horizons , 2018 .

[19]  Elizabeth A. Holm,et al.  A computer vision approach for automated analysis and classification of microstructural image data , 2015 .

[20]  Christopher M Wolverton,et al.  Multiscale modeling of θ′ precipitation in Al–Cu binary alloys , 2004 .

[21]  Fei Meng,et al.  Two-dimensional lithium diffusion behavior and probable hybrid phase transformation kinetics in olivine lithium iron phosphate , 2017, Nature Communications.

[22]  Surya R. Kalidindi,et al.  Structure–property linkages using a data science approach: Application to a non-metallic inclusion/steel composite system , 2015 .

[23]  Xiaolin Li,et al.  A Deep Adversarial Learning Methodology for Designing Microstructural Material Systems , 2018, Volume 2B: 44th Design Automation Conference.

[24]  Markus Gusenbauer,et al.  Magnetic microstructure machine learning analysis , 2018, Journal of Physics: Materials.

[25]  J. Warren,et al.  A general mechanism of polycrystalline growth , 2004, Nature materials.

[26]  Zuhuang Chen,et al.  Kinetic control of tunable multi-state switching in ferroelectric thin films , 2019, Nature Communications.

[27]  Yitong Liu,et al.  A joint deep learning model to recover information and reduce artifacts in missing-wedge sinograms for electron tomography and beyond , 2019, Scientific Reports.

[28]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[29]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[30]  Sergey Levine,et al.  Unsupervised Learning for Physical Interaction through Video Prediction , 2016, NIPS.

[31]  Tim Mueller,et al.  Identifying models of dielectric breakdown strength from high-throughput data via genetic programming , 2017, Scientific Reports.

[32]  Tim Colonius,et al.  FiniteNet: A Fully Convolutional LSTM Network Architecture for Time-Dependent Partial Differential Equations , 2020, ArXiv.

[33]  Nikolas Provatas,et al.  Phase-Field Methods in Materials Science and Engineering , 2010 .

[34]  P. Alam.  ‘S’ , 2021, Composites Engineering: An A–Z Guide.

[35]  Yan Wang,et al.  Data analytics using canonical correlation analysis and Monte Carlo simulation , 2017, npj Computational Materials.

[36]  Kevin M. Ryan,et al.  Crystal Structure Prediction via Deep Learning. , 2018, Journal of the American Chemical Society.

[37]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[38]  Stefano Ermon,et al.  High‐Voltage Charging‐Induced Strain, Heterogeneity, and Micro‐Cracks in Secondary Particles of a Nickel‐Rich Layered Cathode Material , 2019, Advanced Functional Materials.

[39]  Steven L. Brunton,et al.  Data-driven discovery of coordinates and governing equations , 2019, Proceedings of the National Academy of Sciences.

[40]  R. Kobayashi.  Modeling and numerical simulations of dendritic crystal growth , 1993 .

[41]  J. Behler.  Perspective: Machine learning potentials for atomistic simulations. , 2016, The Journal of chemical physics.

[42]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[43]  Shyue Ping Ong,et al.  Deep neural networks for accurate predictions of crystal stability , 2017, Nature Communications.

[44]  Yuksel C. Yabansu,et al.  Material structure-property linkages using three-dimensional convolutional neural networks , 2018 .

[45]  Mario Fritz,et al.  Advanced Steel Microstructural Classification by Deep Learning Methods , 2017, Scientific Reports.

[46]  P. Alam,et al.  R , 1823, The Herodotus Encyclopedia.

[47]  Brian L. DeCost,et al.  Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures , 2017, 1702.01117.

[48]  Chiho Kim,et al.  Machine learning in materials informatics: recent applications and prospects , 2017, npj Computational Materials.

[49]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[50]  Weilin Huang,et al.  V4D: 4D Convolutional Neural Networks for Video-level Representation Learning , 2020, ICLR.

[51]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[52]  W. Mullins.  Two‐Dimensional Motion of Idealized Grain Boundaries , 1956 .

[53]  M. Marques,et al.  Recent advances and applications of machine learning in solid-state materials science , 2019, npj Computational Materials.

[54]  R. Ramesh,et al.  Observation of room-temperature polar skyrmions , 2019, Nature.

[55]  Bart Blanpain,et al.  Quantitative analysis of grain boundary properties in a generalized phase field model for grain growth in anisotropic systems , 2008 .

[56]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[57]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[58]  Jeffrey C Grossman,et al.  Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. , 2017, Physical review letters.

[59]  S. Allen,et al.  Ground state structures in ordered B.C.C. ternary alloys with second-neighbor interactions , 1992 .

[60]  B. Yener,et al.  Image-driven discriminative and generative machine learning algorithms for establishing microstructure-processing relationships , 2020, ArXiv.

[61]  Philip S. Yu,et al.  PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs , 2017, NIPS.

[62]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[63]  Chi Chen,et al.  Random Forest Models for Accurate Identification of Coordination Environments from X-Ray Absorption Near-Edge Structure , 2019, Patterns.

[64]  J. Cahn,et al.  A correction to the ground state of FCC binary ordered alloys with first and second neighbor pairwise interactions , 1973 .

[65]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[66]  Alok Choudhary,et al.  A predictive machine learning approach for microstructure optimization and materials design , 2015, Scientific Reports.

[67]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[68]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[69]  Hu Zhang,et al.  Artificial neural network application to microstructure design of Nb-Si alloy to improve ultimate tensile strength , 2017 .

[70]  Rampi Ramprasad,et al.  Learning scheme to predict atomic forces and accelerate materials simulations , 2015, 1505.02701.

[71]  P. Alam.  ‘N’ , 2021, Composites Engineering: An A–Z Guide.

[72]  Sergei V. Kalinin,et al.  Materials informatics: From the atomic-level to the continuum , 2019, Acta Materialia.

[73]  Yunbo Wang,et al.  Eidetic 3D LSTM: A Model for Video Prediction and Beyond , 2019, ICLR.

[74]  B. Blanpain,et al.  An introduction to phase-field modeling of microstructure evolution , 2008 .

[75]  Maxim Ziatdinov,et al.  Learning surface molecular structures via machine vision , 2017, npj Computational Materials.

[76]  Michele Parrinello,et al.  Generalized neural-network representation of high-dimensional potential-energy surfaces. , 2007, Physical review letters.

[77]  W. Carter,et al.  Branching mechanisms in surfactant micellar growth. , 2012, The journal of physical chemistry. B.

[78]  Cormac Toher,et al.  Universal fragment descriptors for predicting properties of inorganic crystals , 2016, Nature Communications.

[79]  T. Nissen‐Meyer,et al.  Solving the wave equation with physics-informed deep learning , 2020, 2006.11894.

[80]  Daniel Lewis,et al.  An image-driven machine learning approach to kinetic modeling of a discontinuous precipitation reaction , 2019, Materials Characterization.

[81]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[82]  Turab Lookman,et al.  Experimental search for high-temperature ferroelectric perovskites guided by two-step machine learning , 2018, Nature Communications.

[83]  Amir Barati Farimani,et al.  Deep Learning Phase Segregation , 2018, ArXiv.

[84]  Elizabeth Kautz,et al.  A machine learning approach to thermal conductivity modeling: A case study on irradiated uranium-molybdenum nuclear fuels , 2018, Computational Materials Science.

[85]  R. Asahi,et al.  Microstructure recognition using convolutional neural networks for prediction of ionic conductivity in ceramics , 2017 .

[86]  Xiao‐Qing Yang,et al.  Review of Recent Development of In Situ/Operando Characterization Techniques for Lithium Battery Research , 2019, Advanced materials.

[87]  Martin Z. Bazant,et al.  Origin and hysteresis of lithium compositional spatiodynamics within battery primary particles , 2016, Science.

[88]  Masahiko Demura,et al.  Pattern recognition with machine learning on optical microscopy images of typical metallurgical microstructures , 2018, Scientific Reports.

[89]  I. Steinbach.  Phase-field models in materials science , 2009 .

[90]  Robert W. Balluffi,et al.  Kinetics Of Materials , 2005 .

[91]  T. G. Woodcock,et al.  Extracting local nucleation fields in permanent magnets using machine learning , 2019, npj Computational Materials.

[92]  Rama Vasudevan,et al.  Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations. , 2017, ACS nano.

[93]  Peter W. Voorhees,et al.  Optimizing convolutional neural networks to perform semantic segmentation on large materials imaging datasets: X-ray tomography and serial sectioning , 2020, Materials Characterization.

[94]  Daniel W. Davies,et al.  Machine learning for molecular and materials science , 2018, Nature.

[95]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[96]  M. Tang,et al.  Three-dimensional phase evolution and stress-induced non-uniform Li intercalation behavior in lithium iron phosphate , 2020 .

[97]  Wei Chen,et al.  Microstructural Materials Design Via Deep Adversarial Learning Methodology , 2018, Journal of Mechanical Design.

[98]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[99]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[100]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[101]  J. E. Hilliard,et al.  Free Energy of a Nonuniform System. I. Interfacial Free Energy , 1958 .

[102]  Surya R. Kalidindi,et al.  Microstructure informatics using higher-order statistics and efficient data-mining protocols , 2011 .

[103]  David T. Fullwood,et al.  Microstructure Sensitive Design for Performance Optimization , 2012 .

[104]  P. Voorhees,et al.  Characterization of metals in four dimensions , 2020 .

[105]  Philip S. Yu,et al.  PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning , 2018, ICML.

[106]  A. Karma,et al.  Quantitative phase-field modeling of dendritic growth in two and three dimensions , 1996 .