Cohesion Factors: Improving the Clustering Capabilities of Consensus

Security has become a main concern in corporate networks. Security tests are essential to identify vulnerabilities, but experts must analyze very large data and complex information. Unsupervised learning can help by clustering groups of devices with similar vulnerabilities. However an index to evaluate every solution should be calculated to demonstrate results validity. Also the value of the number of clusters should be tuned for every data set in order to find the best solution. This paper introduces SOM as a clustering method to evaluate complex and uncertain knowledge in Consensus, a distributed security system for vulnerability testing; it proposes new metrics to evaluate the cohesion of every cluster, and also the cohesion between clusters; it applies unsupervised algorithms and validity metrics to a security data set; and it presents a method to obtain the best number of clusters regarding these new cohesion metrics: Intracohesion and Intercohesion factors.

[1]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[2]  L.L. DeLooze.  Classification of computer attacks using a self-organizing map , 2004, Proceedings from the Fifth Annual IEEE SMC Information Assurance Workshop, 2004..

[3]  Elisabet Golobardes,et al.  Application of Clustering Techniques in a Network Security Testing System , 2005, CCIA.

[4]  A. Zaballos,et al.  A distributed vulnerability detection system for an intranet , 2005, Proceedings 39th Annual 2005 International Carnahan Conference on Security Technology.

[5]  Guiomar Corral,et al.  A distributed vulnerability detection system for WLANs , 2005, First International Conference on Wireless Internet (WICON'05).

[6]  J. Dunn.  Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[7]  John Hale,et al.  A systematic approach to multi-stage network attack analysis , 2004, Second IEEE International Information Assurance Workshop, 2004. Proceedings..

[8]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[10]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[11]  E. Golobardes,et al.  Unsupervised Case Memory Organization: Analysing Computational Time and Soft Computing Capabilities , 2006, ECCBR.

[12]  E. Bloedorn,et al.  Data mining for network intrusion detection : How to get started , 2001 .

[13]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[14]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[15]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[16]  Christopher Leckie,et al.  Unsupervised Anomaly Detection in Network Intrusion Detection Using Clusters , 2005, ACSC.

[17]  Maurice K. Wong,et al.  Algorithm AS136: A k-means clustering algorithm. , 1979 .