Reduction of False Alarm Rate in Detecting Network Anomaly using Mahalanobis Distance and Similarity Measure

This paper discusses about a network anomaly detection system which is aimed at reduction of the number of false positives and negatives generated by conventional IDSs. A statistical model of the network activities is built using the payload and is trained with the normal behavior of user(s) in the network over a period of time. This model in-turn is used to detect deviations that are high from the expected behavior which indicate a security breach or a possible attack. The payload of the network traffic is analyzed by the system in an unsupervised manner and then classifies as normal traffic during training phase. The value-byte frequency of the application payload is calculated for each normal packet based on payload length and port number. The Mahalanobis distance and a similarity measure is then used to measure the similarity of the incoming data with the already computed values in the detection phase. This distance is then compared against a threshold value and generates an alert if it exceeds the value. In the clustering phase we provide a method to reduce the resource consumption which can easily update the stored profile using an incremental algorithm and the model is continuously updated so that it is accurate. The modeling method that is being followed is completely unsupervised and also tolerant to noise in the training data. The method proposed is also resistant to mimicry-attack. This system is designed to be integrated into other detectors in order to mitigate false positive rates so that this enriches the chances of detecting zero-day worms and new attack exploits

[1]  Philip K. Chan,et al.  Learning Models of Network Traffic for Detecting Novel Attacks , 2002 .

[2]  M Damashek,et al.  Gauging Similarity with n-Grams: Language-Independent Categorization of Text , 1995, Science.

[3]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[4]  Salvatore J. Stolfo,et al.  USENIX Association Proceedings of the FREENIX Track : 2001 USENIX Annual , 2001 .

[5]  V. Rao Vemuri,et al.  Use of K-Nearest Neighbor classifier for intrusion detection , 2002, Comput. Secur..

[6]  Donald E. Knuth,et al.  The art of computer programming: V.1.: Fundamental algorithms , 1997 .

[7]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[8]  Christopher Krügel,et al.  Service specific anomaly detection for network intrusion detection , 2002, SAC '02.

[9]  Salvatore J. Stolfo,et al.  Anomalous Payload-Based Network Intrusion Detection , 2004, RAID.

[10]  David Moore,et al.  Internet quarantine: requirements for containing self-propagating code , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[11]  Sotiris Ioannidis,et al.  Efficient packet monitoring for network management , 2002, NOMS 2002. IEEE/IFIP Network Operations and Management Symposium. ' Management Solutions for the New Communications World'(Cat. No.02CH37327).