Automatic Discovery of Anomalies Reported in Aerospace Systems Health and Safety Documents

Most complex aerospace systems involve large numbers of text reports relating to safety, maintenance, and associated issues. Some have thousands of reports, spanning decades. The Space Shuttle has over 100,000 reports from just the last two decades. Similarly, the Aviation Safety Reporting System (ASRS) database spans several decades and contains over 600,000 reports. These information repositories contain valuable information about system health, particularly about trends and recurring problems. However, repository volume and complexity can make human analysis dicult. Current methods for identifying recurring anomalies rely on a divide-and-conquer strategy. A system is decomposed into subsystems, and experts on those subsystems read and monitor current reports. Thus problems and anomalies can be tracked and trended. This methodology requires experts who can recall and integrate reports spanning potentially long time scales. Current reports are often related to ones from months and years past. Recall is aided by categorizing reports into specific anomaly categories. Tracking and trending are aided by monitoring category totals over time. Category utility, however, relies on the experts to correctly and consistently categorize reports. With multiple experts, inter-rater reliability must somehow be assured. Clearly, human reading, comprehension, and association with relevant prior reports, is essential to system health assurance. The weak point is in identification and recall of relevant prior reports. A decision support system that automatically analyzes reports and provides consistent discovery, characterization, and categorization would be extremely useful. This paper discusses recent innovations in the field of text mining that enable the automatic discovery of anomalies in such text repositories, using statistical and content-based clustering techniques, and the characterization and categorization of these anomalies using advanced classification algorithms. The first innovation discovers recurring anomalies using content-based and statistical clustering techniques. The system, known as the Recurring Anomaly Detection System (ReADS) performs comprehensive analysis of an entire information repository to discover recurring anomalies and presents the results in an intuitive interactive visualization for further investigation by experts. The second innovation, known as Mariana, automatically classifies documents into predetermined categories using an advanced classifier, known as a Support Vector Machine, along with a Markov Chain Monte Carlo simulation to find the best hyperparameters for the model. Another approach to finding predetermined categories, base on Non-negative Matrix Factorization, is also showing promising results. We discuss the application of these to the problems of discovering recurring anomalies and categorizing anomalies in space and aeronautics domains.