An AI-Empowered Framework for Cross-Layer Softwarized Infrastructure State Assessment

Network softwarization technologies challenge legacy fault management systems. Coordination and dependency among different novel software components for orchestration, switching, virtual machine and container management creates novel monitoring points, besides novel sources of faults, bugs and vulnerabilities. To cope with the high heterogeneity and granularity of software components, we propose a modular network AI framework to detect anomalies, toward closed-loop automation. We design an AI-empowered anomaly detection framework able to assess the running state and the state deviations of a softwarized infrastructure, monitored through different features grouped depending on their layer and connect-compute stack component. Our framework learns the nominal working conditions of the infrastructure, with respect to which anomalies are detected, and characterized tracing back the layer and component root cause; it includes a network state assessment technique that leverages anomalies characterization through their most visible symptoms. We implement and validate the proposed framework through experimental tests on a containerized IP Multimedia Subsystem platform.

[1]  Makan Pourzandi,et al.  NFV security survey in 5G networks: A three-dimensional threat taxonomy , 2021, Comput. Networks.

[2]  Raouf Boutaba,et al.  Chronos: DDoS Attack Detection Using Time-Based Autoencoder , 2021, IEEE Transactions on Network and Service Management.

[3]  Stefano Secci,et al.  LSTM-based radiography for anomaly detection in softwarized infrastructures , 2020, 2020 32nd International Teletraffic Congress (ITC 32).

[4]  Fabrice Guillemin,et al.  5G E2E Network Slicing Management with ONAP , 2020, 2020 23rd Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN).

[5]  Fabrice Guillemin,et al.  Network Slice Life-Cycle Management Towards Automation , 2019, 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM).

[6]  Héctor Alaiz-Moretón,et al.  Multiclass Classification Procedure for Detecting Attacks on MQTT-IoT Protocol , 2019, Complex..

[7]  Svetha Venkatesh,et al.  Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  George T. Karetsos,et al.  Traffic forecasting in cellular networks using the LSTM RNN , 2018, PCI.

[9]  Toerless Eckert,et al.  A Reference Model for Autonomic Networking , 2018, RFC.

[10]  Adlen Ksentini,et al.  Improving Traffic Forecasting for 5G Core Network Scalability: A Machine Learning Approach , 2018, IEEE Network.

[11]  Ahmed Meddahi,et al.  NFV Security Survey: From Use Case Driven Threat Analysis to State-of-the-Art Countermeasures , 2018, IEEE Communications Surveys & Tutorials.

[12]  Wolfgang Kellerer,et al.  Assessing the Maturity of SDN Controllers With Software Reliability Growth Models , 2018, IEEE Transactions on Network and Service Management.

[13]  Rashid Mijumbi,et al.  DARN: Dynamic Baselines for Real-time Network Monitoring , 2018, 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft).

[14]  Zhiyong Cui,et al.  Deep Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction , 2018, ArXiv.

[15]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[16]  Peter C. Y. Chen,et al.  LSTM network: a deep learning approach for short-term traffic forecast , 2017 .

[17]  Georgios Xilouris,et al.  Statistical-based anomaly detection for NFV services , 2016, 2016 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN).

[18]  R. Jha,et al.  Anomaly detection in network traffic using K-mean clustering , 2016, 2016 3rd International Conference on Recent Advances in Information Technology (RAIT).

[19]  Enrico Zio,et al.  Resilience-Based Component Importance Measures for Critical Infrastructure Network Systems , 2016, IEEE Transactions on Reliability.

[20]  James P. G. Sterbenz,et al.  Measuring the resilience of mobile ad hoc networks with human walk patterns , 2015, 2015 7th International Workshop on Reliable Networks Design and Modeling (RNDM).

[21]  Filip De Turck,et al.  Network Function Virtualization: State-of-the-Art and Research Challenges , 2015, IEEE Communications Surveys & Tutorials.

[22]  Yunpeng Wang,et al.  Long short-term memory neural network for traffic speed prediction using remote microwave sensor data , 2015 .

[23]  Takehisa Yairi,et al.  Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction , 2014, MLSDA'14.

[24]  T. Chai,et al.  Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature , 2014 .

[25]  Nils-Bastian Heidenreich,et al.  Bandwidth selection for kernel density estimation: a review of fully automatic selectors , 2013, AStA Advances in Statistical Analysis.

[26]  Geoffrey E. Hinton,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[27]  Annie George,et al.  Anomaly Detection based on Machine Learning Dimensionality Reduction using PCA and Classification using SVM , 2012 .

[28]  Gary B. Wills,et al.  Unsupervised Clustering Approach for Network Anomaly Detection , 2012, NDT.

[29]  Alexandru Paler,et al.  Scalable service deployment on software-defined networks , 2011, IEEE Communications Magazine.

[30]  Christos Faloutsos,et al.  Surprising Patterns for the Call Duration Distribution of Mobile Phone Users , 2010, ECML/PKDD.

[31]  Yin Chen,et al.  Statistical anomaly detection with sensor networks , 2010, TOSN.

[32]  David Hutchison,et al.  Resilience and survivability in communication networks: Strategies, principles, and survey of disciplines , 2010, Comput. Networks.

[33]  Qingbo Yang,et al.  A Survey of Anomaly Detection Methods in Networks , 2009, 2009 International Symposium on Computer Network and Multimedia Technology.

[34]  Carolina Fortuna,et al.  Trends in the development of communication networks: Cognitive networks , 2009, Comput. Networks.

[35]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[36]  Martin May,et al.  Applying PCA for Traffic Anomaly Detection: Problems and Solutions , 2009, IEEE INFOCOM 2009.

[37]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[38]  Liang Song,et al.  Cognitive Networks: Standardizing the Large Scale Wireless Systems , 2008, 2008 5th IEEE Consumer Communications and Networking Conference.

[39]  Katherine B. Ensor,et al.  Multivariate Time‐Series Analysis With Categorical and Continuous Variables in an Lstr Model , 2007 .

[40]  Bjarne E. Helvik,et al.  A survey of resilience differentiation frameworks in communication networks , 2007, IEEE Communications Surveys & Tutorials.

[41]  Qusay H. Mahmoud,et al.  Cognitive Networks: Towards Self-Aware Networks , 2007 .

[42]  Jennifer Rexford,et al.  Sensitivity of PCA for traffic anomaly detection , 2007, SIGMETRICS '07.

[43]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[44]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[45]  C. Willmott,et al.  Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance , 2005 .

[46]  Michel Verleysen,et al.  The Curse of Dimensionality in Data Mining and Time Series Prediction , 2005, IWANN.

[47]  David D. Clark,et al.  A knowledge plane for the internet , 2003, SIGCOMM '03.

[48]  B. Kedem,et al.  Regression Theory for Categorical Time Series , 2003 .

[49]  Mary Shaw,et al.  "Self-healing": softening precision to avoid brittleness: position paper for WOSS '02: workshop on self-healing systems , 2002, WOSS '02.

[50]  Stuart Staniford-Chen,et al.  Practical Automated Detection of Stealthy Portscans , 2002, J. Comput. Secur..

[51]  Qiang Chen,et al.  An anomaly detection technique based on a chi‐square statistic for detecting intrusions into information systems , 2001 .

[52]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[53]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[54]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[55]  Jean-Luc Gaudiot,et al.  Network Resilience: A Measure of Network Fault Tolerance , 1990, IEEE Trans. Computers.

[56]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[57]  R. Plackett,et al.  Karl Pearson and the Chi-squared Test , 1983 .

[58]  K. Pearson.  On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[59]  Stefano Secci,et al.  Security and Performance Comparison of ONOS and ODL controllers , 2019 .

[60]  Feng Liu,et al.  A System Architecture for Real-time Anomaly Detection in Large-scale NFV Systems , 2016, FNC/MobiSPC.

[61]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[62]  Sungzoon Cho,et al.  Variational Autoencoder based Anomaly Detection using Reconstruction Probability , 2015 .

[63]  Kostas Pentikousis,et al.  Software-Defined Networking (SDN): Layers and Architecture Terminology , 2015, RFC.

[64]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[65]  Abdul Jabbar,et al.  A framework to quantify network resilience and survivability , 2010 .

[66]  Ryan W. Thomas,et al.  Cognitive networks , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[67]  G. F. Hughes,et al.  On the mean accuracy of statistical pattern recognizers , 1968, IEEE Trans. Inf. Theory.