Random effects logistic regression model for anomaly detection

As the influence of the internet continues to expand as a medium for communications and commerce, the threat from spammers, system attackers, and criminal enterprises has grown accordingly. This paper proposes a random effects logistic regression model to predict anomaly detection. Unlike the previous studies on anomaly detection, a random effects model was applied, which accommodates not only the risk factors of the exposures but also the uncertainty not explained by such factors. The specific factors of the risk category such as retained 'protocol type' and 'logged in' are included in the proposed model. The research is based on a sample of 49,427 random observations for 42 variables of the KDD-cup 1999 (Data Mining and Knowledge Discovery competition) data set that contains 'normal' and 'anomaly' connections. The proposed model has a classification accuracy of 98.94% for the training data set, while that for the validation data set is 98.68%.

[1]  Jan Vanthienen,et al.  A tool-supported approach to inter-tabular verification , 1998 .

[2]  H. Choi,et al.  Random effects logistic regression model for data envelopment analysis with correlated decision making units , 2006, J. Oper. Res. Soc..

[3]  Csilla Farkas,et al.  PAID: A Probabilistic Agent-Based Intrusion Detection system , 2005, Comput. Secur..

[4]  Jing Zhang,et al.  Factor-analysis based anomaly detection and clustering , 2006, Decis. Support Syst..

[5]  M. Shyu,et al.  A Novel Anomaly Detection Scheme Based on Principal Component Classifier , 2003 .

[6]  Ajith Abraham,et al.  Feature deduction and ensemble design of intrusion detection systems , 2005, Comput. Secur..

[7]  So Young Sohn,et al.  RANDOM EFFECTS LINEAR MODELS FOR PROCESS MEAN AND VARIANCE , 1998 .

[8]  Kai Hwang,et al.  Frequent Episode Rules for Intrusive Anomaly Detection with Internet Datamining , 2004 .

[9]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.

[10]  So Young Sohn,et al.  Robust design of server capability in M/M/1 queues with both partly random arrival and service rates , 2002, Comput. Oper. Res..

[11]  Emin Anarim,et al.  An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks , 2005, Expert Syst. Appl..

[12]  So Young Sohn,et al.  Robust parameter design for integrated circuit fabrication procedure with respect to categorical characteristic , 1999 .

[13]  Simon C. K. Shiu,et al.  Formal description and verification of Hybrid Rule/Frame-based Expert Systems , 1997 .

[14]  Jim Alves-Foss,et al.  An empirical analysis of NATE: Network Analysis of Anomalous Traffic Events , 2002, NSPW '02.

[15]  Miguel A. Sanz-Bobi,et al.  DADICC: Intelligent system for anomaly detection in a combined cycle gas turbine plant , 2008, Expert Syst. Appl..

[16]  Ki Hoon Kwon,et al.  DDoS attack detection method using cluster analysis , 2008, Expert Syst. Appl..

[17]  So Young Sohn,et al.  Bayesian dynamic forecasting for attribute reliability , 1997 .

[18]  So Young Sohn.  Statistical Analysis of Environmental Effects on Reliability Decay of Missiles: A Case Study , 1996 .

[19]  Andrew H. Sung,et al.  Intrusion detection using an ensemble of intelligent paradigms , 2005, J. Netw. Comput. Appl..

[20]  Sushil Jajodia,et al.  Detecting Novel Network Intrusions Using Bayes Estimators , 2001, SDM.

[21]  Salvatore J. Stolfo,et al.  A data mining framework for building intrusion detection models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[22]  Erland Jonsson,et al.  Anomaly-based intrusion detection: privacy concerns and other problems , 2000, Comput. Networks.

[23]  Yun Wang,et al.  A multinomial logistic regression modeling approach for anomaly intrusion detection , 2005, Comput. Secur..

[24]  Andrew H. Sung,et al.  Intrusion detection using neural networks and support vector machines , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).