Efficient failure prediction in autonomic networks based on trend and frequency analysis of anomalous patterns

SUMMARY We describe an efficient failure prediction system based on new algorithms that model and detect anomalous behaviors using multi-scale trend analysis of multiple network parameters. Our approach enjoys many advantages over prior approaches. By operating at multiple timescales simultaneously, the new system achieves robustness against unreliable, redundant, incomplete and contradictory information. The algorithms employed operate with low time complexity, making the system scalable and feasible in real-time environments. Anomalous behaviors identified by the system can be stored efficiently with low space complexity, making it possible to operate with minimal resource requirements even when processing high-rate streams of network parameter values. The developed algorithms generate accurate failure predictions quickly, and the system can be deployed in sa distributed setting. Prediction quality can be improved by considering larger sets of network parameters, allowing the approach to scale as network complexity increases. The system is validated by experiments that demonstrate its ability to produce accurate failure predictions in an efficient and scalable manner. Copyright © 2013 John Wiley & Sons, Ltd.

[1]  Marina Thottan,et al.  Anomaly detection in IP networks , 2003, IEEE Trans. Signal Process..

[2]  Martin May,et al.  The autonomic network architecture (ANA) , 2010, IEEE Journal on Selected Areas in Communications.

[3]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[4]  Symeon Papavassiliou,et al.  Implementing enhanced network maintenance for transaction access services: tools and applications , 2000, 2000 IEEE International Conference on Communications. ICC 2000. Global Convergence Through Communications. Conference Record.

[5]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[6]  Frank Feather,et al.  Fault detection in an Ethernet network using anomaly signature matching , 1993, SIGCOMM '93.

[7]  Kavé Salamatian,et al.  Combining filtering and statistical methods for anomaly detection , 2005, IMC '05.

[8]  Takashi Okuda,et al.  Computational intelligence for distributed fault management in networks using fuzzy cognitive maps , 1996, Proceedings of ICC/SUPERCOMM '96 - International Conference on Communications.

[9]  Das Amrita,et al.  Mining Association Rules between Sets of Items in Large Databases , 2013 .

[10]  Lundy M. Lewis,et al.  A case-based reasoning approach to the management of faults in communication networks , 1993, IEEE INFOCOM '93 The Conference on Computer Communications, Proceedings.

[11]  C. S. Hood,et al.  Proactive network-fault detection [telecommunications] , 1997 .

[12]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[13]  B. Sikdar,et al.  Network Management and Control Using Collaborative On-line Simulation , 2001 .

[14]  Robert H. Deng,et al.  Models and algorithms for network fault detection and identification: a review , 1992, [Proceedings] Singapore ICCS/ISITA `92.

[15]  Joachim Sachs,et al.  Ambient networks: an architecture for communication networks beyond 3G , 2004, IEEE Wireless Communications.

[16]  Mark Crovella,et al.  Mining anomalies using traffic feature distributions , 2005, SIGCOMM '05.

[17]  Jeffrey O. Kephart,et al.  The Vision of Autonomic Computing , 2003, Computer.

[18]  Ramesh Govindan,et al.  Detection and identification of network anomalies using sketch subspaces , 2006, IMC '06.

[19]  Levent Yilmaz,et al.  Agent-based simulation study of behavioral anticipation: anticipatory fault management in computer networks , 2006, ACM-SE 44.

[20]  T. Oates Fault Identification in Computer Networks: A Review and a New Approach , 1995 .

[21]  Miroslaw Malek,et al.  Predicting failures of computer systems: a case study for a telecommunication system , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[22]  Mohsen Guizani,et al.  Failure Prediction Based on Multi-Scale Frequent Anomalous Behavior Identification in Support of Autonomic Networks , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[23]  Deep Medhi,et al.  A network management framework for multi-layered network survivability: an overview , 2001, 2001 IEEE/IFIP International Symposium on Integrated Network Management Proceedings. Integrated Network Management VII. Integrated Management Strategies for the New Millennium (Cat. No.01EX470).

[24]  A. Al-Fuqaha,et al.  Prediction of performance degradation in telecommunication networks using Joint Clustering and association analysis techniques , 2010, 2010 IEEE Globecom Workshops.

[25]  Miroslaw Malek,et al.  Using Hidden Semi-Markov Models for Effective Online Failure Prediction , 2007, 2007 26th IEEE International Symposium on Reliable Distributed Systems (SRDS 2007).

[26]  Paul Barford,et al.  A signal analysis of network traffic anomalies , 2002, IMW '02.

[27]  Ali A. Ghorbani,et al.  Network Anomaly Detection Based on Wavelet Analysis , 2009, EURASIP J. Adv. Signal Process..

[28]  Isabelle Rouvellou,et al.  Automatic alarm correlation for fault identification , 1995, Proceedings of INFOCOM'95.

[29]  John S. Baras,et al.  Integrated, Distributed Fault Management for Communication Networks , 1998 .

[30]  Haixun Wang,et al.  Online Anomaly Prediction for Robust Cluster Systems , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[31]  Mydhili K. Nair,et al.  Agent Based Web Services with RuleML for Network Management , 2009, 2009 First International Conference on Networks & Communications.

[32]  Ahmad Aljaafreh,et al.  Failure Prediction based on multi-parameter analysis in support of autonomic networks , 2011, 2011 International Conference on Communications and Information Technology (ICCIT).

[33]  Philippe Owezarski,et al.  UNADA: Unsupervised Network Anomaly Detection Using Sub-space Outliers Ranking , 2011, Networking.

[34]  Mischa Schwartz,et al.  Schemes for fault identification in communication networks , 1995, TNET.

[35]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[36]  Sharma Chakravarthy,et al.  NFM/sup i/: an inner-domain network fault management system , 2005, 21st International Conference on Data Engineering (ICDE'05).

[37]  Peter Fröhlich,et al.  Model-based alarm correlation in cellular phone networks , 1997, Proceedings Fifth International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems.

[38]  Alessandro Bassi,et al.  Management Architecture and Systems for Future Internet Networks , 2009, Future Internet Assembly.

[39]  Dinesh Gambhir,et al.  A framework for adding real-time distributed software fault detection and isolation to SNMP-based systems management , 2005, Journal of Network and Systems Management.

[40]  Carlos Becker Westphall,et al.  Performance evaluation for proactive network management , 1996, Proceedings of ICC/SUPERCOMM '96 - International Conference on Communications.

[41]  Nazim Agoulmine,et al.  FOCALE: A Novel Autonomic Networking Architecture , 2006 .

[42]  Nazim Agoulmine,et al.  Beyond the Knowledge Plane: An Inference Plane to Support the Next Generation Internet , 2007, 2007 First International Global Information Infrastructure Symposium.

[43]  Gabi Dreo Rodosek,et al.  Using the Concept of Intelligent Agents in Fault Management of Distributed Services , 1999, Journal of Network and Systems Management.