User requirements for explanation in expert systems

Explanation facilities are an important part of the expert system (ES) user interface. To be considered useful and acceptable, an ES must be able to present users with explanations of its knowledge of the task domain and the reasoning processes employed to solve problems and make recommendations. While prior work has concentrated on the technology of explanation generation and presentation, little is known about the behavioral impact of such technology. This dissertation explores the user's requirements for ES explanation. A conceptual model of user-ES interaction was developed, which postulated that effective explanation must help the user develop a model of the system compatible with the user's model of the task for which the system is intended. Three generic types of explanations were identified: Trace, Justification, and Control. Level of user expertise in the task domain and type of inference processes involved in diagnostic problem-solving were the user and task characteristics under investigation, and both were hypothesized to have an impact on users' model of the task, and hence on their preference for alternative types of explanation. A controlled experiment was conducted, using a computer program that simulated an ES user interface within the context of conducting analytical reviews in auditing. Twenty auditors, at two levels of expertise, evaluated the conclusions and alternative explanations produced by the program. Both perceptual and behavioral responses were collected to measure the impact of explanation on user preference. The empirical results suggest that ES explanation enhances user understanding of, and confidence in, the system's conclusions and reasoning processes. Justification was found to be the most preferred explanation type across type of inference and level of expertise, and experts and novices were found to value each explanation type differently. This study has the overall implication that user requirements for different types of explanation may be contingent upon the contextual characteristics of the task, and that the benefit of providing a particular type of explanation should be carefully weighed against its cost.