Factor analysis based anomaly detection

We propose a novel anomaly detection algorithm based on factor analysis and Mahalanobis distance. Factor analysis is used to uncover the latent structure (dimensions) of a set of variables. It reduces attribute space from a larger number of variables to a smaller number of factors. The Mahalanobis distance is used to determine the "similarity" of a set of values from an "unknown" sample to a set of values measured from a collection of "known" samples. Combined with factor analysis, Mahalanobis distance is extended to examine whether a given vector is an outlier from a model identified by "factors" based on factor analysis. We present a factor analysis-based network anomaly detection algorithm and apply it to DARPA intrusion detection evaluation data. The experimental results show that the proposed algorithm is able to detect network intrusions with relatively low false alarms.

[1]  Sushil Jajodia,et al.  ADAM: a testbed for exploring the use of data mining in intrusion detection , 2001, SGMD.

[2]  Alfonso Valdes,et al.  Adaptive, Model-Based Monitoring for Cyber Attack Detection , 2000, Recent Advances in Intrusion Detection.

[3]  Rolph E. Anderson,et al.  Multivariate Data Analysis with Readings , 1979 .

[4]  Qiang Chen,et al.  Multivariate Statistical Analysis of Audit Trails for Host-Based Intrusion Detection , 2002, IEEE Trans. Computers.

[5]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[6]  Peter G. Neumann,et al.  Experience with EMERALD to Date , 1999, Workshop on Intrusion Detection and Network Monitoring.

[7]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[8]  Sushil Jajodia,et al.  Detecting Novel Network Intrusions Using Bayes Estimators , 2001, SDM.

[9]  B. Ravichandran,et al.  Statistical traffic modeling for network intrusion detection , 2000, Proceedings 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (Cat. No.PR00728).

[10]  Anup K. Ghosh,et al.  A Study in Using Neural Networks for Anomaly and Misuse Detection , 1999, USENIX Security Symposium.

[11]  R. Sekar,et al.  Specification-based anomaly detection: a new approach for detecting network intrusions , 2002, CCS '02.

[12]  Peter G. Neumann,et al.  EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances , 1997, CCS 2002.