Evaluating performance of long short-term memory recurrent neural networks on intrusion detection data

This paper evaluates the performance of long short-term memory recurrent neural networks (LSTM-RNN) on classifying intrusion detection data. LSTM networks can learn memory and can therefore model data as a time series. LSTM is trained and tested on a processed version of the KDDCup99 dataset. A variety of suitable performance measures are discussed and applied. Our LSTM network structure and parameters are experimentally obtained within a series of experiments presented. Results finally show that LSTM is able to learn all attack classes hidden in the training data. Furthermore we learn that the receiver operating characteristic (ROC) curve and the corresponding area-under-the-curve (AUC) value are well suited for selecting well performing networks.

[1]  Ronald J. Williams,et al.  Experimental Analysis of the Real-time Recurrent Learning Algorithm , 1989 .

[2]  Gürsel Serpen,et al.  Why machine learning algorithms fail in misuse detection on KDD intrusion detection data set , 2004, Intell. Data Anal..

[3]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[4]  Richard Lippmann,et al.  The 1999 DARPA off-line intrusion detection evaluation , 2000, Comput. Networks.

[5]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[6]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[7]  Sugata Sanyal,et al.  Adaptive neuro-fuzzy intrusion detection systems , 2004, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004..

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[12]  Marcus A. Maloof,et al.  Some Basic Concept of Machine Learning and Data Mining , 2006 .

[13]  Salvatore J. Stolfo,et al.  A framework for constructing features and models for intrusion detection systems , 2000, TSEC.

[14]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[15]  Ajith Abraham,et al.  Modeling intrusion detection system using hybrid intelligent systems , 2007, J. Netw. Comput. Appl..

[16]  Lorenzo L. Pesce,et al.  Reliable and computationally efficient maximum-likelihood estimation of "proper" binormal ROC curves. , 2007, Academic radiology.

[17]  Charles Elkan,et al.  Results of the KDD'99 classifier learning , 2000, SKDD.

[18]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[19]  Gürsel Serpen,et al.  Application of Machine Learning Algorithms to KDD Intrusion Detection Dataset within Misuse Detection Context , 2003, MLMTA.

[20]  Andrew H. Sung,et al.  Intrusion detection using neural networks and support vector machines , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[21]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.

[22]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[23]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[24]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .