Cloud monitoring: A survey

Nowadays, Cloud Computing is widely used to deliver services over the Internet for both technical and economical reasons. The number of Cloud-based services has increased rapidly and strongly in the last years, and so is increased the complexity of the infrastructures behind these services. To properly operate and manage such complex infrastructures effective and efficient monitoring is constantly needed. Many works in literature have surveyed Cloud properties, features, underlying technologies (e.g. virtualization), security and privacy. However, to the best of our knowledge, these surveys lack a detailed analysis of monitoring for the Cloud. To fill this gap, in this paper we provide a survey on Cloud monitoring. We start analyzing motivations for Cloud monitoring, providing also definitions and background for the following contributions. Then, we carefully analyze and discuss the properties of a monitoring system for the Cloud, the issues arising from such properties and how such issues have been tackled in literature. We also describe current platforms, both commercial and open source, and services for Cloud monitoring, underlining how they relate with the properties and issues identified before. Finally, we identify open issues, main challenges and future directions in the field of Cloud monitoring.

[1]  Werner Nutt,et al.  R-GMA: An Information Integration System for Grid Monitoring , 2003, OTM.

[2]  Jorge-Arnulfo Quiané-Ruiz,et al.  Runtime measurements in the cloud , 2010, Proc. VLDB Endow..

[3]  Cong Wang,et al.  Security Challenges for the Public Cloud , 2012, IEEE Internet Computing.

[4]  Richard Wolski,et al.  The impact of paravirtualized memory hierarchy on linear algebra computational kernels and software , 2008, HPDC '08.

[5]  Sami Tabbane,et al.  State of the Art and Research Challenges of new services architecture technologies: Virtualization, SOA and Cloud Computing , 2010 .

[6]  Fabio Panzieri,et al.  QoS–Aware Clouds , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[7]  Paul M. Schwartz,et al.  Privacy, Ethics, and Analytics , 2011, IEEE Security & Privacy.

[8]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[9]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[10]  Rosli Salleh,et al.  A Survey on Cloud Computing Security , 2012, ArXiv.

[11]  Lisandro Zambenedetti Granville,et al.  Incorporating virtualization awareness in service monitoring systems , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[12]  Antonio Pescapè,et al.  A customer service assurance platform for mobile broadband networks , 2011, IEEE Communications Magazine.

[13]  Waheed Iqbal,et al.  Adaptive resource provisioning for read intensive multi-tier applications in the cloud , 2011, Future Gener. Comput. Syst..

[14]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[15]  Jianfeng Yang,et al.  Cloud Computing Research and Security Issues , 2010, 2010 International Conference on Computational Intelligence and Software Engineering.

[16]  Marty Humphrey,et al.  A quantitative analysis of high performance computing with Amazon's EC2 infrastructure: The death of the local cluster? , 2009, 2009 10th IEEE/ACM International Conference on Grid Computing.

[17]  Muli Ben-Yehuda,et al.  The Reservoir model and architecture for open federated cloud computing , 2009, IBM J. Res. Dev..

[18]  Jean-Claude Laprie,et al.  From Dependability to Resilience , 2008, DSN 2008.

[19]  Carlos Becker Westphall,et al.  Toward an architecture for monitoring private clouds , 2011, IEEE Communications Magazine.

[20]  Edward Walker,et al.  Benchmarking Amazon EC2 for High-Performance Scientific Computing , 2008, login Usenix Mag..

[21]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[22]  Jonathan M. Spring,et al.  Monitoring Cloud Computing by Layer, Part 1 , 2011, IEEE Security & Privacy.

[23]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[24]  Roland Kübert,et al.  Building a Service-Oriented Monitoring Framework with REST and Nagios , 2011, 2011 IEEE International Conference on Services Computing.

[25]  Robert W. Shirey,et al.  Internet Security Glossary , 2000, RFC.

[26]  Salvatore Venticinque,et al.  Cloud Application Monitoring: The mOSAIC Approach , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[27]  Raouf Boutaba,et al.  A survey of network virtualization , 2010, Comput. Networks.

[28]  Eddy Caron,et al.  Auto-Scaling, Load Balancing and Monitoring in Commercial and Open-Source Clouds , 2011 .

[29]  Xing Pu,et al.  Performance Measurements and Analysis of Network I/O Applications in Virtualized Cloud , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[30]  Aleksander Slominski,et al.  A Business Centric End-to-End Monitoring Approach for Service Composites , 2010, 2010 IEEE International Conference on Services Computing.

[31]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[32]  Ji Su Park,et al.  Markov Chain Based Monitoring Service for Fault Tolerance in Mobile Cloud Computing , 2011, 2011 IEEE Workshops of International Conference on Advanced Information Networking and Applications.

[33]  T. S. Eugene Ng,et al.  The Impact of Virtualization on Network Performance of Amazon EC2 Data Center , 2010, 2010 Proceedings IEEE INFOCOM.

[34]  Chonho Lee,et al.  A survey of mobile cloud computing: architecture, applications, and approaches , 2013, Wirel. Commun. Mob. Comput..

[35]  Benny Rochwerger,et al.  Reservoir - When One Cloud Is Not Enough , 2011, Computer.

[36]  Sergio Andreozzi,et al.  GridICE: a monitoring service for Grid systems , 2005, Future Gener. Comput. Syst..

[37]  Hovav Shacham,et al.  Hey, you, get off of my cloud: exploring information leakage in third-party compute clouds , 2009, CCS.

[38]  A. Fox,et al.  Cloudstone : Multi-Platform , Multi-Language Benchmark and Measurement Tools for Web 2 . 0 , 2008 .

[39]  Shufen Zhang,et al.  Cloud Computing Research and Development Trend , 2010, 2010 Second International Conference on Future Networks.

[40]  Virgílio A. F. Almeida,et al.  Capacity Planning for Web Services: Metrics, Models, and Methods , 2001 .

[41]  Iosif Legrand,et al.  MonALISA : A Distributed Monitoring Service Architecture , 2003, ArXiv.

[42]  Alysson Neves Bessani.  From Byzantine fault tolerance to intrusion tolerance (a position paper) , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks Workshops (DSN-W).

[43]  Shilpashree Srinivasamurthy,et al.  Survey on Cloud Computing Security , 2010 .

[44]  Xiaowei Yang,et al.  CloudCmp: comparing public cloud providers , 2010, IMC '10.

[45]  Guillaume Pierre,et al.  EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications , 2009, ICSOC/ServiceWave Workshops.

[46]  Wei Chen,et al.  Short Text Feature Selection for Micro-Blog Mining , 2010, 2010 International Conference on Computational Intelligence and Software Engineering.

[47]  Mohiuddin Ahmed,et al.  An Advanced Survey on Cloud Computing and State-of-the-art Research Issues , 2012 .

[48]  Jennifer G. Dy,et al.  Workload Characterization at the Virtualization Layer , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[49]  Nick Feamster,et al.  Broadband internet performance: a view from the gateway , 2011, SIGCOMM.

[50]  Anders Eriksson,et al.  What Networking of Information Can Do for Cloud Computing , 2009, 2009 18th IEEE International Workshops on Enabling Technologies: Infrastructures for Collaborative Enterprises.

[51]  Marcelo Pasin,et al.  Trustworthy and resilient monitoring system for cloud infrastructures , 2011, PDT '11.

[52]  Rajkumar Buyya,et al.  Towards autonomic detection of SLA violations in Cloud infrastructures , 2012, Future Gener. Comput. Syst..

[53]  Ahmed Patel,et al.  Review of pricing models for grid & cloud computing , 2011, 2011 IEEE Symposium on Computers & Informatics.

[54]  A. B. M. Shawkat Ali,et al.  A survey on gaps, threat remediation challenges and some thoughts for proactive attack detection in cloud computing , 2012, Future Gener. Comput. Syst..

[55]  Rajkumar Buyya,et al.  High-Performance Cloud Computing: A View of Scientific Applications , 2009, 2009 10th International Symposium on Pervasive Systems, Algorithms, and Networks.

[56]  Jin Shao,et al.  A Performance Guarantee Approach for Cloud Applications Based on Monitoring , 2011, 2011 IEEE 35th Annual Computer Software and Applications Conference Workshops.

[57]  Luigi Atzori,et al.  A Network-Oriented Survey and Open Issues in Cloud Computing , 2011 .

[58]  Alexandru Iosup,et al.  A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing , 2009, CloudComp.

[59]  Antonio Pescapè,et al.  MagNets - experiences from deploying a joint research-operational next-generation wireless access network testbed , 2007, 2007 3rd International Conference on Testbeds and Research Infrastructure for the Development of Networks and Communities.

[60]  Archana Ganapathi,et al.  Statistics-driven workload modeling for the Cloud , 2010, 2010 IEEE 26th International Conference on Data Engineering Workshops (ICDEW 2010).

[61]  Jonathan M. Spring,et al.  Monitoring Cloud Computing by Layer, Part 2 , 2011, IEEE Security & Privacy.

[62]  Randy H. Katz,et al.  Above the Clouds: A Berkeley View of Cloud Computing , 2009 .

[63]  Massimo Villari,et al.  Achieving Federated and Self-Manageable Cloud Infrastructures: Theory and Practice , 2012 .

[64]  Georgios Nakos,et al.  Monitoring , 1976, Encyclopedia of the UN Sustainable Development Goals.

[65]  Anas Ayad,et al.  Agent-based monitoring of virtual machines , 2010, 2010 International Symposium on Information Technology.

[66]  Antonio Pescapè,et al.  A tool for the generation of realistic network workload for emerging networking scenarios , 2012, Comput. Networks.

[67]  José Luis Vázquez-Poletti,et al.  Provisioning data analytic workloads in a cloud , 2013, Future Gener. Comput. Syst..

[68]  Carsten Binnig,et al.  How is the weather tomorrow?: towards a benchmark for the cloud , 2009, DBTest '09.

[69]  Archana Ganapathi,et al.  Towards Understanding Cloud Performance Tradeoffs Using Statistical Workload Analysis and Replay , 2010 .

[70]  Rajkumar Buyya,et al.  Aneka: a Software Platform for .NET based Cloud Computing , 2009, High Performance Computing Workshop.

[71]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[72]  George Pavlou,et al.  Monitoring, aggregation and filtering for efficient management of virtual networks , 2011, 2011 7th International Conference on Network and Service Management.

[73]  Nico d'Heureuse,et al.  Towards holistic multi-tenant monitoring for virtual data centers , 2010, 2010 IEEE/IFIP Network Operations and Management Symposium Workshops.

[74]  Indranil Gupta,et al.  Performance Evaluation of the Illinois Cloud Computing Testbed , 2009 .

[75]  Renata Teixeira,et al.  Explaining packet delays under virtualization , 2011, CCRV.

[76]  Dimitrios Zissis,et al.  Addressing cloud computing security issues , 2012, Future Gener. Comput. Syst..

[77]  Ankit Shah,et al.  Network-based Measurements on Cloud Computing Services , 2008 .

[78]  Stuart Clayman,et al.  Monitoring virtual networks with Lattice , 2010, 2010 IEEE/IFIP Network Operations and Management Symposium Workshops.

[79]  Yanpei Chen,et al.  What's New About Cloud Computing Security? , 2010 .

[80]  Antonio Corradi,et al.  DDS-enabled Cloud management support for fast task offloading , 2012, 2012 IEEE Symposium on Computers and Communications (ISCC).

[81]  John Shalf,et al.  Characterizing and predicting the I/O performance of HPC applications using a parameterized synthetic benchmark , 2008, HiPC 2008.

[82]  Poulami Dalapati.  A Survey on Cloud Computing , 2013 .

[83]  Gurdev Singh,et al.  CM Measurement Facets for Cloud Performance , 2011 .

[84]  virtualization.info 日本語.  論文：The Impact of Virtualization on Network Performance of Amazon EC2 Data Center（20100601-1） , 2010 .

[85]  Devarshi Ghoshal,et al.  I/O performance of virtualized cloud environments , 2011, DataCloud-SC '11.

[86]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[87]  Antonio Pescapè,et al.  Quality of service statistics over heterogeneous networks: Analysis and applications , 2008, Eur. J. Oper. Res..

[88]  Irfan-Ullah Awan,et al.  Performance Evaluation of Local and Cloud Deployment of Web Clusters , 2011, 2011 14th International Conference on Network-Based Information Systems.

[89]  Rizos Sakellariou,et al.  A taxonomy of grid monitoring systems , 2005, Future Gener. Comput. Syst..

[90]  Jin Shao,et al.  A Runtime Model Based Monitoring Approach for Cloud , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[91]  Benny Rochwerger,et al.  A Monitoring and Audit Logging Architecture for Data Location Compliance in Federated Cloud Infrastructures , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[92]  Rajkumar Buyya,et al.  The Aneka platform and QoS-driven resource provisioning for elastic applications on hybrid Clouds , 2012, Future Gener. Comput. Syst..

[93]  Willy Zwaenepoel,et al.  Performance profiling of virtual machines , 2011, VEE '11.

[94]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[95]  A. Johnsson,et al.  Time-stamping accuracy in virtualized environments , 2011, 13th International Conference on Advanced Communication Technology (ICACT2011).

[96]  T. Chiueh,et al.  A Survey on Virtualization Technologies , 2005 .

[97]  Yoshihiro Kawahara,et al.  On demand fine grain resource monitoring system for server consolidation , 2010, 2010 ITU-T Kaleidoscope: Beyond the Internet? - Innovations for Future Networks and Services.

[98]  Rajkumar Buyya,et al.  A dependency‐aware ontology‐based approach for deploying service level agreement monitoring services in Cloud , 2012, Softw. Pract. Exp..

[99]  Sherif Abdelwahed,et al.  Large Scale Monitoring and Online Analysis in a Distributed Virtualized Environment , 2011, 2011 Eighth IEEE International Conference and Workshops on Engineering of Autonomic and Autonomous Systems.

[100]  Tim Kraska,et al.  An evaluation of alternative architectures for transaction processing in the cloud , 2010, SIGMOD Conference.

[101]  Karsten Schwan,et al.  Implementing Diverse Messaging Models with Self-Managing Properties using IFLOW , 2006, 2006 IEEE International Conference on Autonomic Computing.

[102]  Vanish Talwar,et al.  A flexible architecture integrating monitoring and analytics for managing large-scale data centers , 2011, ICAC '11.

[103]  Joy Bhattacharjee,et al.  A Survey on Cloud Computing Security, Challenges and Threats , 2011 .

[104]  Ian Lumb,et al.  A Taxonomy and Survey of Cloud Computing Systems , 2009, 2009 Fifth International Joint Conference on INC, IMS and IDC.

[105]  Qiang Zhang,et al.  The Characteristics of Cloud Computing , 2010, 2010 39th International Conference on Parallel Processing Workshops.

[106]  Vanish Talwar,et al.  Monalytics: online monitoring and analytics for managing large scale data centers , 2010, ICAC '10.

[107]  Hai Jin,et al.  VMDriver: A Driver-Based Monitoring Mechanism for Virtualization , 2010, 2010 29th IEEE Symposium on Reliable Distributed Systems.

[108]  Massimo Bernaschi,et al.  Analysis and experimentation over heterogeneous wireless networks , 2005, First International Conference on Testbeds and Research Infrastructures for the DEvelopment of NeTworks and COMmunities.

[109]  Christof Fetzer,et al.  A Novel Approach to QoS Monitoring in the Cloud , 2011, 2011 First International Conference on Data Compression, Communications and Processing.