Clustering IT Events around Common Root Causes

This paper focuses on clustering alerts around common root causes at the lower levels of the event management chain. The aim is to enable root-cause identification from a mixed event stream and to offer aggregated information for holistic problem solving. This end-to-end investigation spans feature selection and similarity assessment, clustering on heterogeneous feature maps, and evaluation of results. We compare feature values based on network information, user-defined similarity matrices, and textual analysis, and capture aspects of feature correlation in the event similarity function. Spectral clustering partitions the stream and serves to learn a more general similarity metric from a reference partitioning. Finally, we introduce two novel result visualization techniques and make a case study on one identified root-cause for which this framework outperforms both a time-pressured human operator and baseline clustering algorithms.

[1]  Polly Bart,et al.  Heuristic Methods for Estimating the Generalized Vertex Median of a Weighted Graph , 1968, Oper. Res..

[2]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[3]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[4]  Bin Yu,et al.  Spectral clustering and the high-dimensional stochastic blockmodel , 2010, 1007.1684.

[5]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[6]  Daniela Rosu,et al.  Multi-dimensional Knowledge Integration for Efficient Incident Management in a Services Cloud , 2009, 2009 IEEE International Conference on Services Computing.

[7]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[8]  Francis R. Bach,et al.  Large-Margin Metric Learning for Partitioning Problems , 2013, ArXiv.

[9]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[10]  Peter J. Rousseeuw,et al.  Clustering by means of medoids , 1987 .

[11]  T. Snijders,et al.  Estimation and Prediction for Stochastic Blockstructures , 2001 .

[12]  Michael I. Jordan,et al.  Learning Spectral Clustering, With Application To Speech Separation , 2006, J. Mach. Learn. Res..

[13]  Raymond J. Mooney,et al.  Integrating constraints and metric learning in semi-supervised clustering , 2004, ICML.

[14]  Ling Huang,et al.  Fast approximate spectral clustering , 2009, KDD.

[15]  Dirk Husemann,et al.  Automatic Classification of Change Requests for Improved IT Service Quality , 2011, 2011 Annual SRII Global Conference.

[16]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[17]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..