True and false dependence on technology: Evaluation with an expert system

Abstract There is a danger inherent in labeling systems “expert.” Such identification implies some levels of “intelligence” or “understanding” within the confines of the system. It is important to know the limitations of any system, including realistic expectations of the real or implied power of an expert system. The “blindness” or boundaries inherent in expert system development extends to users who may misplace trust in false technology. This study investigates the use of an incorrect advice-giving expert system. Expert and novice engineers used the faulty system to solve a well test interpretation task. Measures of decision confidence, system success, state-anxiety and task difficulty were taken. Subjects expressed confidence in their “wrong” answer to the problem, displaying true dependence on a false technology. These results suggest implications for developers and/or users in areas of certification, evaluation, risk assessment, validation, and verification of systems conveying a level of “expertise.”