Policy-Based QoS Enforcement for Adaptive Big Data Distribution on the Cloud

Big Data distribution has benefited from the Cloud resources to accommodate application's QoS requirements. In this paper, we propose Big Data distribution scheme that matches the Cloud available resources to guarantee application's QoS given the continuously dynamic and varying resources of the Cloud infrastructure. We developed Two-Level QoS Policies (TLPS) for selecting clusters and nodes while satisfying the client's application QoS. We also proposed an adaptive data distribution algorithm to cope with changing QoS requirements. Experiments have been conducted to evaluate both the effectiveness and the communication overhead of our proposed distribution scheme and the results we have reported are convincing. Other experiments evaluated our TLPS algorithm against other single-based QoS data distribution algorithms and the results show that TLPS algorithm adapts to the customer QoS requirements.