On the automated assessment of nuclear reactor systems code accuracy

Abstract An automated code assessment program (ACAP) has been developed to provide quantitative comparisons between nuclear reactor systems (NRS) code results and experimental measurements. The tool provides a suite of metrics for quality of fit to specific data sets, and the means to produce one or more figures of merit (FOM) for a code, based on weighted averages of results from the batch execution of a large number of code–experiment and code–code data comparisons. Accordingly, this tool has the potential to significantly streamline the verification and validation (V and V) processes in NRS code development environments which are characterized by rapidly evolving software, many contributing developers and a large and growing body of validation data. In this paper, a survey of data conditioning and analysis techniques is summarized which focuses on their relevance to NRS code accuracy assessment. A number of methods are considered for their applicability to the automated assessment of the accuracy of NRS code simulations. A variety of data types and computational modeling methods are considered from a spectrum of mathematical and engineering disciplines. The goal of the survey was to identify needs, issues and techniques to be considered in the development of an automated code assessment procedure, to be used in United States Nuclear Regulatory Commission (NRC) advanced thermal–hydraulic T/H code consolidation efforts. The ACAP software was designed based in large measure on the findings of this survey. An overview of this tool is summarized and several NRS data applications are provided. The paper is organized as follows: The motivation for this work is first provided by background discussion that summarizes the relevance of this subject matter to the nuclear reactor industry. Next, the spectrum of NRS data types are classified into categories, in order to provide a basis for assessing individual comparison methods. Then, a summary of the survey is provided, where each of the relevant issues and techniques considered are addressed. Several of the methods have been coded and/or applied to relevant NRS code–data comparisons and these demonstration calculations are included. Next, an overview of the basic design, structure and operational mechanics of ACAP is provided. Then, a summary of the data pre-processing, data analysis and FOM assembly processing elements of the software is included. Lastly, a number of NRS sample applications are presented which illustrate the functionality of the code and its ability to provide objective accuracy measures.

[1]  Hugh W. Coleman,et al.  Uncertainties and CFD Code Validation , 1997 .

[2]  R. W. Shumway.  Assessment of MIT and UCB wall condensation tests and of the pre-release RELAP5/MOD3.2 code condensation models , 1995 .

[3]  Julius S. Bendat,et al.  Engineering Applications of Correlation and Spectral Analysis , 1980 .

[4]  Francesco Saverio D'Auria,et al.  Current Status of Methodologies evaluating the Uncertainties in the prediction of thermalhydraulic phenomena in nuclear reactors , 1995 .

[5]  J. H. Jo,et al.  Assessment of TRAC-BD1/MOD1 using FIST data , 1985 .

[6]  John H. Mahaffy,et al.  A Review of Data Analysis Techniques for Application in Automated Quantitative Accuracy Assessments , 2000 .

[7]  Numerical simulation of air pollution in urban areas: model performance , 1987 .

[8]  Douglas G. Fox,et al.  Uncertainty in Air Quality Modeling , 1984 .

[9]  John S. Irwin,et al.  Evaluation of the performance of RAM with the Regional Air Pollution Study data base , 1985 .

[10]  John S. Irwin,et al.  Resampling and extreme value statistics in air quality model performance evaluation , 1985 .

[11]  Joseph G. Marvin,et al.  Perspective on Computational Fluid Dynamics Validation , 1995 .

[12]  Jon S. Simons,et al.  2D/3D Program work summary report, [January 1988--December 1992] , 1993 .

[13]  R. Kunz,et al.  TECHNIQUES TO NUCLEAR REACTOR SYSTEMS CODE ACCURACY ASSESSMENT , 2000 .

[14]  L. N. Kmetyk,et al.  Methodology for code accuracy quantification , 1985 .

[15]  C. Willmott.  Some Comments on the Evaluation of Model Performance , 1982 .

[16]  R. Kunz,et al.  An Automated Code Assessment Program for Determining Systems Code Accuracy , 2000 .

[17]  W. Ambrosini,et al.  Evaluation of accuracy of thermal hydraulic code calculation , 1990 .

[18]  F. D'Auria,et al.  User effects on the thermal-hydraulic transient system code calculations , 1993 .

[19]  Francesco Saverio D'Auria,et al.  A proposed Methodology for the Analysis of a Phenomenon in Separate Effects and Integral Test Facilities , 1995 .

[20]  Francesco Saverio D'Auria,et al.  Code Assessment Methodology and Results , 1990 .

[21]  Francesco Saverio D'Auria,et al.  Code validation and uncertainties in system thermalhydraulics , 1998 .

[22]  S. Hanna,et al.  Air quality model performance evaluation , 2004 .

[23]  Francesco Saverio D'Auria,et al.  Scaling of the accuracy of the Relap5/mod2 code , 1993 .

[24]  Francesco Saverio D'Auria,et al.  A methodology for the qualification of thermalhydraulic codes Nodalizations , 1992 .

[25]  J. D. Burtt,et al.  Development and application of methods to characterize code uncertainty , 1985 .

[26]  R. R. Schultz.  International Code Assessment and Applications Program: Summary of code assessment studies concerning RELAP5/MOD2, RELAP5/MOD3, and TRAC-B. International Agreement Report , 1993 .

[27]  F. D'auria,et al.  Outline of the uncertainty methodology based on accuracy extrapolation , 1995 .

[28]  Douglas G. Fox,et al.  Judging Air Quality Model Performance , 1981 .