Predictive Synthesis of Quantum Materials by Probabilistic Reinforcement Learning

Predictive materials synthesis is the primary bottleneck in realizing new functional and quantum materials. Strategies for synthesis of promising materials are currently identified by time-consuming trial and error approaches and there are no known predictive schemes to design synthesis parameters for new materials. We use reinforcement learning to predict optimal synthesis schedules, i.e. a time-sequence of reaction conditions like temperatures and reactant concentrations, for the synthesis of a prototypical quantum material, semiconducting monolayer MoS$_{2}$, using chemical vapor deposition. The predictive reinforcement leaning agent is coupled to a deep generative model to capture the crystallinity and phase-composition of synthesized MoS$_{2}$ during CVD synthesis as a function of time-dependent synthesis conditions. This model, trained on 10000 computational synthesis simulations, successfully learned threshold temperatures and chemical potentials for the onset of chemical reactions and predicted new synthesis schedules for producing well-sulfidized crystalline and phase-pure MoS$_{2}$, which were validated by computational synthesis simulations. The model can be extended to predict profiles for synthesis of complex structures including multi-phase heterostructures and can also predict long-time behavior of reacting systems, far beyond the domain of the MD simulations used to train the model, making these predictions directly relevant to experimental synthesis.

[1]  Koray Kavukcuoglu,et al.  Pixel Recurrent Neural Networks , 2016, ICML.

[2]  Stefanie Jegelka,et al.  Virtual screening of inorganic materials synthesis parameters with deep learning , 2017, npj Computational Materials.

[3]  Magnus Rueping,et al.  Self-Optimizing Reactor Systems: Algorithms, On-line Analytics, Setups, and Strategies for Accelerating Continuous Flow Process Optimization , 2014 .

[4]  Vladan Stevanović,et al.  Material descriptors for predicting thermoelectric performance , 2015 .

[5]  Noam Bernstein,et al.  De novo exploration and self-guided learning of potential-energy surfaces , 2019, npj Computational Materials.

[6]  Chiho Kim,et al.  Machine learning in materials informatics: recent applications and prospects , 2017, npj Computational Materials.

[7]  Ichiro Takeuchi,et al.  Fulfilling the promise of the materials genome initiative with high-throughput experimental methodologies , 2017 .

[8]  Krishna Rajan,et al.  New frontiers for the materials genome initiative , 2019, npj Computational Materials.

[9]  Richard N. Zare,et al.  Optimizing Chemical Reactions with Deep Reinforcement Learning , 2017, ACS central science.

[10]  Zhijian Ou,et al.  A Review of Learning with Deep Generative Models from perspective of graphical modeling , 2018, ArXiv.

[11]  Alán Aspuru-Guzik,et al.  Accelerating the discovery of materials for clean energy in the era of smart automation , 2018, Nature Reviews Materials.

[12]  Li Li,et al.  Decoding Molecular Graph Embeddings with Reinforcement Learning , 2019, ArXiv.

[13]  Priya Vashishta,et al.  Defect Healing in Layered Materials: A Machine Learning-Assisted Characterization of MoS2 Crystal Phases. , 2019, The journal of physical chemistry letters.

[14]  Klavs F Jensen,et al.  Integrated microreactors for reaction automation: new approaches to reaction development. , 2010, Annual review of analytical chemistry.

[15]  Ryan P. Adams,et al.  Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach. , 2016, Nature materials.

[16]  A. Zunger Inverse design in search of materials with target functionalities , 2018 .

[17]  Qian Wang,et al.  Phase-selective synthesis of 1T′ MoS2 monolayers and heterophase bilayers , 2018, Nature Materials.

[18]  Regina Barzilay,et al.  Prediction of Organic Reaction Outcomes Using Machine Learning , 2017, ACS central science.

[19]  Pieter P. Plehiers,et al.  A robotic platform for flow synthesis of organic compounds informed by AI planning , 2019, Science.

[20]  Qian Yang,et al.  Learning reduced kinetic Monte Carlo models of complex chemistry from molecular dynamics , 2017, Chemical science.

[21]  Alán Aspuru-Guzik,et al.  Neural Networks for the Prediction of Organic Chemistry Reactions , 2016, ACS central science.

[22]  Si-Young Choi,et al.  Atomically thin three-dimensional membranes of van der Waals semiconductors by wafer-scale growth , 2019, Science Advances.

[23]  J. E. Gubernatis,et al.  Machine learning in materials design and discovery: Examples from the present and suggestions for the future , 2018, Physical Review Materials.

[24]  Rajiv K. Kalia,et al.  A scalable parallel algorithm for large-scale reactive force-field molecular dynamics simulations , 2008, Comput. Phys. Commun..

[25]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[26]  Piyush M. Tagade,et al.  Attribute driven inverse materials design using deep learning Bayesian framework , 2019, npj Computational Materials.

[27]  Alán Aspuru-Guzik,et al.  Optimizing distributions over molecular space. An Objective-Reinforced Generative Adversarial Network for Inverse-design Chemistry (ORGANIC) , 2017 .

[28]  Rajiv K. Kalia,et al.  RXMD: A scalable reactive molecular dynamics simulator for optimized time-to-solution , 2020, SoftwareX.

[29]  Priya Vashishta,et al.  Computational Synthesis of MoS2 Layers by Reactive Molecular Dynamics Simulations: Initial Sulfidation of MoO3 Surfaces. , 2017, Nano letters.

[30]  Alán Aspuru-Guzik,et al.  Inverse molecular design using machine learning: Generative models for matter engineering , 2018, Science.

[31]  Olexandr Isayev,et al.  Deep reinforcement learning for de novo drug design , 2017, Science Advances.

[32]  Emma Strubell,et al.  Machine-learned and codified synthesis parameters of oxide materials , 2017, Scientific Data.

[33]  Gerbrand Ceder,et al.  Text-mined dataset of inorganic materials synthesis recipes , 2019, Scientific Data.

[34]  Taylor D. Sparks,et al.  Data-Driven Review of Thermoelectric Materials: Performance and Resource Considerations , 2013 .

[35]  Hugo Larochelle,et al.  The Neural Autoregressive Distribution Estimator , 2011, AISTATS.

[36]  Rajiv K. Kalia,et al.  A Reactive Molecular Dynamics Study of Atomistic Mechanisms During Synthesis of MoS_2 Layers by Chemical Vapor Deposition , 2018 .

[37]  Mike Preuss,et al.  Planning chemical syntheses with deep neural networks and symbolic AI , 2017, Nature.

[38]  Paul Raccuglia,et al.  Machine-learning-assisted materials discovery using failed experiments , 2016, Nature.

[39]  Kristin A. Persson,et al.  Efficient Discovery of Optimal N-Layered TMDC Hetero-Structures , 2018, MRS Advances.

[40]  Daan Wierstra,et al.  Deep AutoRegressive Networks , 2013, ICML.

[41]  Rajiv K. Kalia,et al.  An extended-Lagrangian scheme for charge equilibration in reactive molecular dynamics simulations , 2015, Comput. Phys. Commun..

[42]  Rajiv K. Kalia,et al.  Chemical Vapor Deposition Synthesis of MoS2 Layers from the Direct Sulfidation of MoO3 Surfaces Using Reactive Molecular Dynamics Simulations , 2018 .

[43]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[44]  Magnus Rueping,et al.  Self‐Optimizing Reactor Systems: Algorithms, On‐Line Analytics, Setups, and Strategies for Accelerating Continuous Flow Process Optimization , 2014 .

[45]  Nir Friedman,et al.  Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning , 2009 .

[46]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[47]  Daniel W. Davies,et al.  Machine learning for molecular and materials science , 2018, Nature.

[48]  Sharon C Glotzer,et al.  Efficient Phase Diagram Sampling by Active Learning , 2018, 1803.03296.

[49]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.