Self-supervised learning for an operational knowledge-based system

Abstract This paper addresses the problems of designing a learning system for a diagnostic knowledge based system (KBS) operating in industry. Key features considered are: learning from noisy data; incremental learning as the database expands from empty; learning from cases where multiple faults are present; and reliable learning for an autonomous learning system in a working environment. Starting from the problems of the target application, a Bayesian learning system is developed, incorporating careful assessment of candidate knowledge at data level and knowledge base update level. Knowledge base updating by two methods: Learning and Assessing on the Same Set of Data, and Learning and Assessing on Disjoint Sets of Data are examined quantitatively. Results from the learning system upon application to databases of simulated KBS data are presented. Initially, no data from the target KBS was available. Consequently the system has not undergone domain specific ‘tuning’. The results presented are produced by a system based on only the most basic assumptions. It is shown that the learning system can reliably improve on knowledge elicited under ideal conditions, and that assessing candidate knowledge on a disjoint set of cases greatly improves reliability. It is shown that very large improvements can be obtained against poor quality elicited knowledge.