Advancing Fusion with Machine Learning Research Needs Workshop Report

Machine learning and artificial intelligence (ML/AI) methods have been used successfully in recent years to solve problems in many areas, including image recognition, unsupervised and supervised classification, game-playing, system identification and prediction, and autonomous vehicle control. Data-driven machine learning methods have also been applied to fusion energy research for over 2 decades, including significant advances in the areas of disruption prediction, surrogate model generation, and experimental planning. The advent of powerful and dedicated computers specialized for large-scale parallel computation, as well as advances in statistical inference algorithms, have greatly enhanced the capabilities of these computational approaches to extract scientific knowledge and bridge gaps between theoretical models and practical implementations. Large-scale commercial success of various ML/AI applications in recent years, including robotics, industrial processes, online image recognition, financial system prediction, and autonomous vehicles, have further demonstrated the potential for data-driven methods to produce dramatic transformations in many fields. These advances, along with the urgency of need to bridge key gaps in knowledge for design and operation of reactors such as ITER, have driven planned expansion of efforts in ML/AI within the US government and around the world. The Department of Energy (DOE) Office of Science programs in Fusion Energy Sciences (FES) and Advanced Scientific Computing Research (ASCR) have organized several activities to identify best strategies and approaches for applying ML/AI methods to fusion energy research. This paper describes the results of a joint FES/ASCR DOE-sponsored Research Needs Workshop on Advancing Fusion with Machine Learning, held April 30–May 2, 2019, in Gaithersburg, MD (full report available at https://science.osti.gov/-/media/fes/pdf/workshop-reports/FES_ASCR_Machine_Learning_Report.pdf ). The workshop drew on broad representation from both FES and ASCR scientific communities, and identified seven Priority Research Opportunities (PRO’s) with high potential for advancing fusion energy. In addition to the PRO topics themselves, the workshop identified research guidelines to maximize the effectiveness of ML/AI methods in fusion energy science, which include focusing on uncertainty quantification, methods for quantifying regions of validity of models and algorithms, and applying highly integrated teams of ML/AI mathematicians, computer scientists, and fusion energy scientists with domain expertise in the relevant areas.

[1]  Howie Choset,et al.  Expensive Function Optimization with Stochastic Binary Outcomes , 2013, ICML.

[2]  David Allan Humphreys,et al.  Chapter 8: Plasma operation and control , 2007 .

[3]  Janet Kelso,et al.  The genome of the offspring of a Neandertal mother and a Denisovan father , 2018, Nature.

[4]  Jan S. Hesthaven,et al.  Structure Preserving Model Reduction of Parametric Hamiltonian Systems , 2017, SIAM J. Sci. Comput..

[5]  A. Werner,et al.  Large Scale Bayesian Data Analysis for Nuclear Fusion Experiments , 2007, 2007 IEEE International Symposium on Intelligent Signal Processing.

[6]  Pieter Abbeel,et al.  Learning for control from multiple demonstrations , 2008, ICML '08.

[7]  N. W. Eidietis,et al.  Disruptions in ITER and strategies for their control and mitigation , 2015 .

[8]  E A Baltz,et al.  Achievement of Sustained Net Plasma Heating in a Fusion Experiment with the Optometrist Algorithm , 2017, Scientific Reports.

[9]  E. M. Campbell,et al.  Tripled yield in direct-drive laser fusion through statistical modelling , 2019, Nature.

[10]  J. Gaffney,et al.  Making inertial confinement fusion models more predictive , 2019, Physics of Plasmas.

[11]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[12]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Belinda B. King,et al.  Proper orthogonal decomposition for reduced basis feedback controllers for parabolic equations , 2001 .

[14]  B. P. Duval,et al.  Design and first applications of the ITER integrated modelling & analysis suite , 2015 .

[15]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[16]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[17]  J. A. Leuer,et al.  Tokamak disruption alarm based on a neural network model of the high- beta limit , 1997 .

[18]  Todd A. Oliver,et al.  Solving Irregular and Data-enriched Differential Equations using Deep Neural Networks , 2019, ArXiv.

[19]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[21]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[22]  N. W. Eidietis,et al.  Machine learning for disruption warnings on Alcator C-Mod, DIII-D, and EAST , 2019, Nuclear Fusion.

[23]  Kaj Nyström,et al.  Data-driven discovery of PDEs in complex datasets , 2018, J. Comput. Phys..

[24]  S. A. Lazerson.  Three-dimensional equilibrium reconstruction on the DIII-D device , 2015 .

[25]  D. A. Humphreys,et al.  Novel aspects of plasma control in ITER , 2013 .

[26]  Dave Osthus,et al.  Prediction Uncertainties beyond the Range of Experience: A Case Study in Inertial Confinement Fusion Implosion Experiments , 2019, SIAM/ASA J. Uncertain. Quantification.

[27]  Patrick Riley,et al.  Three pitfalls to avoid in machine learning , 2019, Nature.

[28]  Julian Kates-Harbeck,et al.  Predicting disruptive instabilities in controlled fusion plasmas through deep learning , 2019, Nature.

[29]  R. Schaback,et al.  On generalized moving least squares and diffuse derivatives , 2012 .

[30]  Philip H. Carns,et al.  Efficient I/O and Storage of Adaptive-Resolution Data , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[31]  Ralph C. Smith,et al.  Uncertainty Quantification: Theory, Implementation, and Applications , 2013 .

[32]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[33]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[34]  Jet Efda Contributors,et al.  A cross-tokamak neural network disruption predictor for the JET and ASDEX Upgrade tokamaks , 2005 .

[35]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[36]  Charles Hansen,et al.  The Visualization Handbook , 2011 .

[37]  N. W. Eidietis,et al.  Disruption prediction investigations using Machine Learning tools on DIII-D and Alcator C-Mod , 2018, Plasma Physics and Controlled Fusion.

[38]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[39]  Brian D. Wirth,et al.  Data-driven material models for atomistic simulation , 2019, Physical Review B.

[40]  Anuj Karpatne,et al.  Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling , 2017, ArXiv.

[41]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[42]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[43]  V. Igochine,et al.  Active Control of Magneto-hydrodynamic Instabilities in Hot Plasmas , 2015 .

[44]  Egemen Kolemen,et al.  A Riccati Solution for the Ideal MHD Plasma Response with Applications to Real-time Stability Control , 2018 .

[45]  R. E. Bell,et al.  A reduced resistive wall mode kinetic stability model for disruption forecasting , 2017 .

[46]  Prabhat,et al.  Exascale Deep Learning for Climate Analytics , 2018, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis.

[47]  Kelli Humbird,et al.  Zonal flow generation in inertial confinement fusion implosions , 2017 .

[48]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[49]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[50]  Valerio Pascucci,et al.  Scalable Visualization and Interactive Analysis Using Massive Data Streams , 2012, High Performance Computing Workshop.

[51]  Mark D. Boyer,et al.  Machine learning control for disruption and tearing mode avoidance , 2020 .

[52]  Sudip K. Seal,et al.  Helical core reconstruction of a DIII-D hybrid scenario tokamak discharge , 2017 .

[53]  Pieter Abbeel,et al.  An Application of Reinforcement Learning to Aerobatic Helicopter Flight , 2006, NIPS.

[54]  Neoklis Polyzotis,et al.  Data Management Challenges in Production Machine Learning , 2017, SIGMOD Conference.

[55]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[56]  Sudip K. Seal,et al.  Parallel Reconstruction of Three Dimensional Magnetohydrodynamic Equilibria in Plasma Confinement Devices , 2017, 2017 46th International Conference on Parallel Processing (ICPP).

[57]  D. Higdon,et al.  Computer Model Calibration Using High-Dimensional Output , 2008 .

[58]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[59]  Maciej Haranczyk,et al.  Capturing chemical intuition in synthesis of metal-organic frameworks , 2019, Nature Communications.