A Comparative Study of Different Methods for Representing and Reasoning with Uncertainty for a Waste Characterization Task

Containers of transuranic and low-level alpha contamin ted waste generated as a byproduct of Department of Energy defense-related programs must be char acterized before their proper disposition can be determined. Although nondestructive as say methods are the primary method for assessing the mass and activity of the entrained t ransuranic radionuclides, there are additional sources of useful information relevant to the characte ization of the entrained waste. These include known strengths and weaknesses of assay systems, expected correlations between assay measurements, container manifests, information about the generation process, and destructive assay techniques performed on representative samples. Ea ch of these sources of information provides evidence that may confirm or refute the charac te ization of the materials as determined by the assay system(s). Many different types of uncer tainty and vagueness are associated with the different types of evidence. This paper describes a co mparative study and evaluation of four different uncertainty calculi for representing and reason ing with the uncertainty and/or vagueness associated with evidence in this domain. These methods are: MYCIN-style certainty factors, Dempster-Shafer Theory, Bayesian networks, and fuzzy lo gic. Primary factors considered in the evaluation were power and “naturalness” of the represen tatio for different types of evidence, applicability of the reasoning methods to this domain, the theoretical basis for the representation and reasoning methods, utility of the method for decision making, scale-up considerations for large systems, and knowledge acquisition issues. 1.0 Introduction Human experts are quite adept at reasoning with uncertain, incomplete, and imprecise information. Expert systems which are built to emulat e human reasoning must also be able to represent and reason with this type of information. T here have been many heated debates in the artificial intelligence (AI) literature about the effi cacy of different methods for reasoning with uncertainty (e.g. Elkan 1994; Bezdek 1994). Almost all of thes e di cussions concede that determination of the appropriate methods must be made on t he basis of the properties of the expert task under consideration, but few studies have been reported that compare the applicability of different methods to specific problems or classes of problems. The problem under consideration here is the nondestructive characterizat ion of containerized radioactive waste; in particular determining whether the waste meets all of t he criteria necessary for it to be shipped to the Waste Isolation Pilot Plant (WIPP) permanent sto rage facility in New Mexico. Nondestructive assay methods are available to determine the mass of radionuclides in the waste containers. An uncertainty measurement is associated w ith these masses that is based on the counting statistics of the assay system. There are, however, other known sources of uncertainty that are associated with the measurements. For exampl e, some matrix materials are known to interfere with the measurements, certain configuration s f the radioactive materials in the containers are known to cause problems, etc. In some cas s, additional assays were done at the time the waste was packaged that may be more or less ac curate than nondestructive assay information now available. There is also ancillary information available about the processes that were used to produce the waste that can provide confirmati on of the identity of the matrix material and/or radionuclides involved. Our goal is to build an expert system that can integrate d a from multiple sources to confirm or refute the characterization of the waste material and to quantify the confidence in that characterization. Many different methods for represen ting uncertainty/vagueness in expert systems have been developed (see Kanal and Lemmer 1986 for a n overview), but the four general methods are widely accepted and used are MYCIN-style certa inty factors, Dempster-Shafer theory, Bayesian networks, and fuzzy logic (Henkin and Ha rrison 1988; Stefik 1995). All of these methods involve a quantitative representation of uncerta inty. Methods that use qualitative representations of uncertainty have also been developed, but these were not considered because quantification of the uncertainty associated with the c aracterization is one of the requirements for this task. In order to evaluate the applicability of different meth ods for representing and reasoning with uncertainty for the waste characterization task, we have developed a set of criteria for evaluating the methods, have analyzed the types of uncert ain information that need to be represented, have determined how experts combine the diff erent types of information when solving this task, and are in the process of developing pro totypes using the different representation methods to empirically evaluate the effe ctiveness of each method. Section 2 of this paper provides a brief description of the waste character iza ion task under consideration. Section 3 will describe the evaluation criteria that were devel op d, will enumerate and illustrate the different types of uncertainty that we have found to be important in this domain, and will describe different ways that experts combine evidence when solv ing this problem. Section 4 will give a brief description of the different methods for represent ing uncertainty that have been investigated, will describe how each of the types of uncertainty iden tifi d in the domain can potentially be represented with each method, and will describe combinat ion operators provided for each method. Section 5 summarizes the results and describes future work in development of the expert system. 2.0 The Waste Characterization Task INEL and MSU are cooperating to design and build an expert s ystem called WAMIS (Waste Assay Measurement Integration System). The go al of WAMIS is to improve the confidence in the characterization of containerized r a iological waste based on a variety of data such as gamma spectra; radionuclide mass estimates; total alph activity; thermal power; real-time radiography video; container attributes; and mass ratio e stimates for americium, plutonium, and uranium isotopes. Figure 1 illustrates the types of infor mation that will be combined by the system. In its simplest form, the problem being addressed is the classification of containers of radiological waste into one of two categories: those that can be shipped to a permanent storage site and those that cannot. Classification, which i s a traditional task for knowledge-based systems, may be defined as identifying classes of data a s solutions to a problem (Stefik 1995) . For example, in our application, the data comes from a w ide variety of sources that include simple measurements (e.g., the weight of a drum of waste), clas sific tions based on human judgment (e.g., determination of the type of waste matrix based on real-time radiography),