A data analytics flow typically operates on three layers: ingestion, analytics, and storage, each of which is provided by a data-intensive system. These systems are often available as cloud managed services, enabling the users to have pain-free deployment of data analytics flow applications such as click-stream analytics. Despite straightforward orchestration, elasticity management of the flows is challenging. This is due to: a) heterogeneity of workloads and diversity of cloud resources such as queue partitions, compute servers and NoSQL throughputs capacity, b) workload dependencies between the layers, and c) different performance behaviours and resource consumption patterns.
In this demonstration, we present Flower, a holistic elasticity management system that exploits advanced optimization and control theory techniques to manage elasticity of complex data analytics flows on clouds. Flower analyzes statistics and data collected from different data-intensive systems to provide the user with a suite of rich functionalities, including: workload dependency analysis, optimal resource share analysis, dynamic resource provisioning, and cross-platform monitoring. We will showcase various features of Flower using a real-world data analytics flow. We will allow the audience to explore Flower by visually defining and configuring a data analytics flow elasticity manager and get hands-on experience with integrated data analytics flow management.
[1]
Magdalena Balazinska,et al.
PerfEnforce Demonstration: Data Analytics with Performance Guarantees
,
2016,
SIGMOD Conference.
[2]
Rajiv Ranjan,et al.
Dimensions for Evaluating Cloud Resource Orchestration Frameworks
,
2016,
Computer.
[3]
Kalyanmoy Deb,et al.
A fast and elitist multiobjective genetic algorithm: NSGA-II
,
2002,
IEEE Trans. Evol. Comput..
[4]
Jeffrey S. Chase,et al.
Automated control for elastic storage
,
2010,
ICAC '10.
[5]
Kang G. Shin,et al.
Adaptive control of virtualized resources in utility computing environments
,
2007,
EuroSys '07.
[6]
Mor Harchol-Balter,et al.
Saving Cash by Using Less Cache
,
2012,
HotCloud.
[7]
Ioannis Konstantinou,et al.
TIRAMOLA: elastic nosql provisioning through a cloud management platform
,
2012,
SIGMOD Conference.
[8]
Rajiv Ranjan,et al.
Elasticity management of Streaming Data Analytics Flows on clouds
,
2017,
J. Comput. Syst. Sci..