Anomaly detection in flight recorder data: A dynamic data-driven approach

This paper presents a method of feature extraction in the context of aviation data analysis. The underlying algorithm utilizes a feature extraction algorithm called symbolic dynamic filtering (SDF) that was recently published. In SDF, time-series data are partitioned for generating symbol sequences that, in turn, construct probabilistic finite state automata (PFSA) to serve as features for pattern classification. The SDF-based algorithm of feature extraction, which enjoys both flexibility of implementation and computational efficiency, is directly applicable to detection, classification, and prediction of anomalies and faults. The results of analysis with real-world flight recorder data show that the SDF-based features can be derived at a desired level of abstraction from the information embedded in the time-series data. The performance of the proposed SDF-based feature extraction is compared with that of standard temporal feature extraction for anomaly detection. Our study on flight recorder data shows that SDF-based features can enable discovering unique anomalous flights and improve the performance of the detection algorithm. We also theoretically show that under certain conditions it may be possible to achive a better or comparable time complexity with SDF based features.

[1]  David L. Iverson Inductive System Health Monitoring , 2004, IC-AI.

[2]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[3]  Kanishka Bhaduri,et al.  Algorithms for speeding up distance-based outlier detection , 2011, KDD.

[4]  Irving C. Statler,et al.  The Aviation System Monitoring and Modeling (ASMM) Project: A Documentation of its History and Accomplishments: 1999-2005 , 2007 .

[5]  Maxime Gariel,et al.  Anomaly detection in onboard-recorded flight data using cluster analysis , 2011, 2011 IEEE/AIAA 30th Digital Avionics Systems Conference.

[6]  Stephen D. Bay,et al.  Mining distance-based outliers in near linear time with randomization and a simple pruning rule , 2003, KDD '03.

[7]  Ashok N. Srivastava,et al.  Anomaly Detection and Diagnosis Algorithms for Discrete Symbol Sequences with Applications to Airline Safety , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[9]  Xin Jin,et al.  Symbolic Dynamic Filtering and Language Measure for Behavior Identification of Mobile Robots , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Asok Ray,et al.  Symbolic dynamic analysis of complex systems for anomaly detection , 2004, Signal Process..

[11]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[12]  Ashok N. Srivastava,et al.  Multiple kernel learning for heterogeneous anomaly detection: algorithm and aviation safety case study , 2010, KDD.

[13]  Asok Ray,et al.  Data-Driven Fault Detection in Aircraft Engines With Noisy Sensor Measurements , 2011 .

[14]  A. Ray,et al.  Space partitioning via Hilbert transform for symbolic time series analysis , 2008 .

[15]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[16]  Asok Ray,et al.  Symbolic time series analysis via wavelet-based partitioning , 2006, Signal Process..