Monitoring Data Reduction in Data Centers: A Correlation-Based Approach

Monitoring data are collected and stored in a wide range of domains, especially in data centers, which integrate myriads of services and massive data. To handle the inevitable challenges brought by increasing volume of monitoring data, this paper proposes a correlation-based reduction method for streaming data that derives quantitative formulas between correlated indicators, and reduces the sampling rate of some indicators by replacing them with formulas predictions. This approach also revises formulas through iterations of the reduction process to find an adaptive solution in dynamic environments of data centers. One highlight of this work is the ability to work on upstream side, i.e., it can reduce volume requirements for data collection of monitoring systems. This work also tests the approach with both simulated and real data, showing that our approach is capable of data reduction in complex data centers.

[1]  Barbara Pernici,et al.  Learning a goal-oriented model for energy efficient adaptive applications in data centers , 2015, Inf. Sci..

[2]  Barbara Pernici,et al.  Correlation-model-based reduction of monitoring data in data centers , 2016, 2016 5th International Conference on Smart Cities and Green ICT Systems (SMARTGREENS).

[3]  Chi-Sheng Shih,et al.  Supporting Service Adaptation in Fault Tolerant Internet of Things , 2015, 2015 IEEE 8th International Conference on Service-Oriented Computing and Applications (SOCA).

[4]  Xuesong Peng.  Data Reduction in Monitored Data , 2015, CAiSE.

[5]  Eamonn J. Keogh,et al.  Dimensionality Reduction for Fast Similarity Search in Large Time Series Databases , 2001, Knowledge and Information Systems.

[6]  Nazim Agoulmine,et al.  Improving Prediction Accuracy for WSN Data Reduction by Applying Multivariate Spatio-Temporal Correlation , 2011, Sensors.

[7]  Qiang Fu,et al.  YADING: Fast Clustering of Large-Scale Time Series Data , 2015, Proc. VLDB Endow..

[8]  H. T. Kung,et al.  CloudSense: Continuous Fine-Grain Cloud Monitoring with Compressive Sensing , 2011, HotCloud.

[9]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[10]  I. Jolliffe.  Principal Component Analysis , 2002 .

[11]  Suman Nath,et al.  Managing Massive Time Series Streams with MultiScale Compressed Trickles , 2009, Proc. VLDB Endow..

[12]  Carlos Agón,et al.  Time-series data mining , 2012, CSUR.

[13]  Nikolay Mehandjiev,et al.  On Achieving Energy Efficiency and Reducing CO2 Footprint in Cloud Computing , 2016, IEEE Transactions on Cloud Computing.

[14]  Kalyan Veeramachaneni,et al.  Modeling Service Execution on Data Centers for Energy Efficiency and Quality of Service Monitoring , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[15]  Barbara Pernici,et al.  CO2-Aware Adaptation Strategies for Cloud Applications , 2016, IEEE Transactions on Cloud Computing.