An incremental anomaly detection model for virtual machines

Self-Organizing Map (SOM) algorithm as an unsupervised learning method has been applied in anomaly detection due to its capabilities of self-organizing and automatic anomaly prediction. However, because of the algorithm is initialized in random, it takes a long time to train a detection model. Besides, the Cloud platforms with large scale virtual machines are prone to performance anomalies due to their high dynamic and resource sharing characters, which makes the algorithm present a low accuracy and a low scalability. To address these problems, an Improved Incremental Self-Organizing Map (IISOM) model is proposed for anomaly detection of virtual machines. In this model, a heuristic-based initialization algorithm and a Weighted Euclidean Distance (WED) algorithm are introduced into SOM to speed up the training process and improve model quality. Meanwhile, a neighborhood-based searching algorithm is presented to accelerate the detection time by taking into account the large scale and high dynamic features of virtual machines on cloud platform. To demonstrate the effectiveness, experiments on a common benchmark KDD Cup dataset and a real dataset have been performed. Results suggest that IISOM has advantages in accuracy and convergence velocity of anomaly detection for virtual machines on cloud platform.

[1]  Juan Julián Merelo Guervós,et al.  A novel representation of genomic sequences for taxonomic clustering and visualization by means of self-organizing maps , 2015, Bioinform..

[2]  Toby Berger,et al.  A software-only videocodec using pixelwise conditional differential replenishment and perceptual enhancements , 1999, IEEE Trans. Circuits Syst. Video Technol..

[3]  V. Lakshmikantham,et al.  Stability of conditionally invariant sets and controlleduncertain dynamic systems on time scales , 1995 .

[4]  Xuelong Li,et al.  Spectral-Spatial Constraint Hyperspectral Image Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Vanish Talwar,et al.  Statistical techniques for online anomaly detection in data centers , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[6]  Jaeyoung Choi,et al.  Towards an integrated management system based on abstraction of heterogeneous virtual resources , 2014, Cluster Computing.

[7]  Juha Vesanto,et al.  SOM-based data visualization methods , 1999, Intell. Data Anal..

[8]  Pasi Koikkalainen,et al.  Progress with the Tree-Structured Self-Organizing Map , 1994, ECAI.

[9]  Gaochao Xu,et al.  A Heuristic Placement Selection of Live Virtual Machine Migration for Energy-Saving in Cloud Computing Environment , 2014, PloS one.

[10]  Qian Yang,et al.  Minimizing Average Startup Latency of VMs by an Optimized VM Templates Caching Mechanism Based on K-Medoids Clustering in an IaaS System with Multi-cluster of Servers , 2015 .

[11]  R. Chavez-Arroyo,et al.  Statistical–dynamical downscaling of wind fields using self-organizing maps , 2015 .

[12]  James A. Reggia,et al.  Self-organizing maps based on limit cycle attractors , 2015, Neural Networks.

[13]  Xu-Dong Zhang,et al.  Learning to Rank from Noisy Data , 2015, ACM Trans. Intell. Syst. Technol..

[14]  A. Rzhetsky,et al.  Self-Correcting Maps of Molecular Pathways , 2006, PloS one.

[15]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[16]  Junqing Yu,et al.  Wide area localization and tracking on camera phones for mobile augmented reality systems , 2015, Multimedia Systems.

[17]  Julio Ortega Lopera,et al.  PCA filtering and probabilistic SOM for network intrusion detection , 2015, Neurocomputing.

[18]  George E. Gooden,et al.  Benchmarking Undedicated Cloud Computing Providers for Analysis of Genomic Datasets , 2014, bioRxiv.

[19]  Jianxin Li,et al.  CyberLiveApp: A secure sharing and migration approach for live virtual desktop applications in a cloud environment , 2013, Future Gener. Comput. Syst..

[20]  Solahuddin Shamsuddin,et al.  An overview of neural networks use in anomaly Intrusion Detection Systems , 2009, 2009 IEEE Student Conference on Research and Development (SCOReD).

[21]  Zheng Yuan,et al.  A New Algorithm Combining Self Organizing Map with Simulated Annealing Used in Intrusion Detection , 2009, 2009 2nd International Conference on Biomedical Engineering and Informatics.

[22]  Yu Song,et al.  Fault diagnosis and process monitoring using a statistical pattern framework based on a self-organizing map , 2015 .

[23]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[24]  Hai Jin,et al.  CCAP: A Cache Contention-Aware Virtual Machine Placement Approach for HPC Cloud , 2013, International Journal of Parallel Programming.

[25]  José Carlos Príncipe,et al.  Self-organizing maps with information theoretic learning , 2015, Neurocomputing.

[26]  Gabriele C. Hegerl,et al.  Relating changes in synoptic circulation to the surface rainfall response using self-organising maps , 2015, Climate Dynamics.

[27]  Michael I. Jordan,et al.  Statistical Machine Learning Makes Automatic Control Practical for Internet Datacenters , 2009, HotCloud.

[28]  Tak-Lon Wu,et al.  Cloud computing paradigms for pleasingly parallel biomedical applications , 2011, Concurr. Comput. Pract. Exp..

[29]  Bernd Fritzke.  Growing self-organizing networks—history, status quo, and perspectives , 1999 .

[30]  Kusum Deep,et al.  A new hybrid Self Organizing Migrating Genetic Algorithm for function optimization , 2007, 2007 IEEE Congress on Evolutionary Computation.

[31]  Daniel Polani.  On the Optimization of Self-Organizing Maps by Genetic Algorithms , 1999 .

[32]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[33]  Jian Zhang,et al.  Melton: a practical and precise memory leak detection tool for C programs , 2014, Frontiers of Computer Science.

[34]  Francesco Corona,et al.  Regional models: A new approach for nonlinear system identification via clustering of the self-organizing map , 2015, Neurocomputing.

[35]  Alfons Juan-Císcar,et al.  Comparison of Four Initialization Techniques for the K -Medians Clustering Algorithm , 2000, SSPR/SPR.

[36]  Song Fu,et al.  Adaptive Anomaly Identification by Exploring Metric Subspace in Cloud Computing Infrastructures , 2013, 2013 IEEE 32nd International Symposium on Reliable Distributed Systems.

[37]  Mohammad Ali Riahi,et al.  Seismic facies analysis from pre-stack data using self-organizing maps , 2014 .

[38]  T. Villmann,et al.  Topology Preservation in Self-Organizing Maps , 1999 .

[39]  Emin Germen,et al.  Anomaly Detection with Self-Organizing Maps and Effects of Principal Component Analysis on Feature Vectors , 2009, 2009 Fifth International Conference on Natural Computation.

[40]  Antonio Martínez-Álvarez,et al.  Feature selection by multi-objective optimisation: Application to network anomaly detection by hierarchical self-organising maps , 2014, Knowl. Based Syst..

[41]  Hugh P. Shanahan,et al.  Bioinformatics on the Cloud Computing Platform Azure , 2014, PloS one.

[42]  Pasi Koikkalainen,et al.  Self-organizing hierarchical feature maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[43]  Rongrong Ji,et al.  On-Device Mobile Landmark Recognition Using Binarized Descriptor with Multifeature Fusion , 2015, ACM Trans. Intell. Syst. Technol..

[44]  Jose M. Alcaraz Calero,et al.  MonPaaS: An Adaptive Monitoring Platformas a Service for Cloud Computing Infrastructures and Services , 2015, IEEE Trans. Serv. Comput..

[45]  Takeshi Yamakawa,et al.  Binary Self-Organizing Map with Modified Updating Rule and Its Application to Reproduction of Genetic Algorithm , 2007, IEICE Trans. Inf. Syst..

[46]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[47]  Liujuan Cao,et al.  Joint Depth and Semantic Inference from a Single Image via Elastic Conditional Random Field , 2016, Pattern Recognit..

[48]  Shuyu Chen,et al.  An Anomaly Detection Algorithm of Cloud Platform Based on Self-Organizing Maps , 2016 .

[49]  Esa Alhoniemi,et al.  Clustering of the self-organizing map , 2000, IEEE Trans. Neural Networks Learn. Syst..

[50]  Jun Liu,et al.  Anomaly-based Intrusion Detection using Multiclass-SVM with Parameters Optimized by PSO , 2015 .

[51]  Mohamed Medhat Gaber,et al.  An efficient Self-Organizing Active Contour model for image segmentation , 2015, Neurocomputing.

[52]  Yan Zhang,et al.  Inertial sensors supported visual descriptors encoding and geometric verification for mobile visual location recognition applications , 2015, Signal Process..

[53]  Erkki Oja,et al.  Application of tree structured self-organizing maps in content-based image retrieval , 1999 .

[54]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Man Lan,et al.  Initialization of cluster refinement algorithms: a review and comparative study , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[56]  Dejan S. Milojicic,et al.  OpenNebula: A Cloud Management Tool , 2011, IEEE Internet Computing.

[57]  Xiaohui Gu,et al.  PREPARE: Predictive Performance Anomaly Prevention for Virtualized Cloud Systems , 2012, 2012 IEEE 32nd International Conference on Distributed Computing Systems.

[58]  Michele Scardi,et al.  Applications of Self-Organizing Maps for Ecomorphological Investigations through Early Ontogeny of Fish , 2014, PloS one.

[59]  Chu-Hsing Lin,et al.  Anomaly Detection Using LibSVM Training Tools , 2008, 2008 International Conference on Information Security and Assurance (isa 2008).

[60]  Gustavo J. Meschino,et al.  Automatic design of interpretable fuzzy predicate systems for clustering using self-organizing maps , 2015, Neurocomputing.

[61]  Yin Chen,et al.  Statistical anomaly detection with sensor networks , 2010, TOSN.

[62]  Tommy W. S. Chow,et al.  Self-Organizing and Self-Evolving Neurons: A New Neural Network for Optimization , 2007, IEEE Transactions on Neural Networks.

[63]  Geoffrey C. Fox,et al.  Cloud computing paradigms for pleasingly parallel biomedical applications , 2010, HPDC '10.

[64]  A. Skupin,et al.  Visualizing the Topical Structure of the Medical Sciences: A Self-Organizing Map Approach , 2013, PloS one.