Improving Fault Diagnosis Performance Using Hadoop MapReduce for Efficient Classification and Analysis of Large Data Sets

Underpinning a significant amount of the mass quantities of data, virtualization technology is a key element of utility cloud and an area in which monitoring is a special challenge. The monitoring of large, complex systems requires high accuracy, low latency, and near-real-time fault detection and anomaly analysis along with optimization enactment and actions for corrections. For this paper, we investigated a fine-grained fault-tolerance mechanism with newly proposed algorithms for the analysis of large datasets that are based on the Hadoop MapReduce platform, and we implement a Naïve Bayes Classifier (NBC) algorithm with Hadoop MapReduce to achieve high-performance and efficient classification for the analysis procedure that occurs in virtualization and utility cloud. Evaluation results show that the accuracy of our proposed method using Hadoop MapReduce approaches 89.80% as the size of the data sets increases. We demonstrate that our model is scalable to large data sets of virtual machine (VM) component utilization metrics with increased accuracy, low latency, and machine learning ability.