Application of Machine Learning Algorithms to Identify Problematic Nuclear Data

Abstract In this work, we aim to show that machine learning algorithms are promising tools for the identification of nuclear data that contribute to increased errors in transport simulations. We demonstrate this through an application of a machine learning algorithm (Random Forest) to the Whisper/MCNP6 criticality validation library to identify nuclear data that are associated with an increase of the bias (simulated-experimental ) in the calculations. Specifically, the sensitivity profiles (with respect to nuclear data) of solution benchmarks are used to predict the bias, and SHapley Additive exPlanations (SHAP) are used to explain how the sensitivities are related to the predicted bias. The SHAP values can be interpreted as sensitivity coefficients of the machine learning model to the sensitivities that are used to make predictions of bias. Using the SHAP values, we can identify specific subsets of nuclear data that have the highest probability of influencing bias. We demonstrate the utility of this method by showing how SHAP values were used to identify an inconsistency in the inelastic scattering nuclear data. The methodology presented here is not limited to transport problems and can be applied to other simulations if there are experimental measurements to compare against, simulations of those experimental measurements, and the ability to calculate sensitivities of the model output with respect to the data inputs.

[1]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[2]  J. B. Briggs,et al.  The International Criticality Safety Benchmark Evaluation Project , 2002 .

[3]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[4]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[5]  N. M. Larson,et al.  ENDF/B-VII.1 Nuclear Data for Science and Technology: Cross Sections, Covariances, Fission Product Yields and Decay Data , 2011 .

[6]  R. Q. Wright,et al.  ENDF/B-VIII.0: The 8 th Major Release of the Nuclear Reaction Data Library with CIELO-project Cross Sections, New Standards and Thermal Scattering Data , 2018 .

[7]  B. Rearden,et al.  Sensitivity- and Uncertainty-Based Criticality Safety Validation Techniques , 2004 .

[8]  Forrest B. Brown,et al.  User Manual for Whisper-1.1 , 2017 .

[9]  Todd S. Palmer,et al.  Using Machine Learning Methods to Predict Bias in Nuclear Criticality Safety , 2018, Journal of Computational and Theoretical Transport.

[10]  K. Shibata,et al.  JENDL-4.0: A New Library for Nuclear Science and Engineering , 2011 .

[11]  Hugh Chen,et al.  From local explanations to global understanding with explainable AI for trees , 2020, Nature Machine Intelligence.

[12]  John S. Hendricks,et al.  Initial MCNP6 Release Overview , 2012 .

[13]  Adrian F. M. Smith,et al.  A Bayesian CART algorithm , 1998 .

[14]  B. Kiedrowski.  Review of Early 21st-Century Monte Carlo Perturbation and Sensitivity Techniques for k-Eigenvalue Radiation Transport Calculations , 2017 .

[15]  Forrest B. Brown,et al.  Methodology, verification, and performance of the continuous-energy nuclear data sensitivity capability in MCNP6 , 2013 .

[16]  Bianca Zadrozny,et al.  Learning and evaluating classifiers under sample selection bias , 2004, ICML.

[17]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[18]  B. Rearden.  Perturbation Theory Eigenvalue Sensitivity Analysis with Monte Carlo Techniques , 2004 .

[19]  Patrick Talou,et al.  Enhancing nuclear data validation analysis by using machine learning , 2020 .

[20]  Forrest B. Brown,et al.  Whisper: Sensitivity/Uncertainty-Based Computational Methods and Software for Determining Baseline Upper Subcritical Limits , 2015 .

[21]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[22]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[23]  Prabhat,et al.  Graph Neural Networks for Particle Reconstruction in High Energy Physics detectors , 2020, 2003.11603.