SOPHIA: Online Reconfiguration of Clustered NoSQL Databases for Time-Varying Workloads

Reconfiguring NoSQL databases in the face of changing workload patterns is crucial for maximizing database throughput. However, this is challenging because of the large configuration parameter search space with complex interdependencies among the parameters. While state-of-the-art systems can automatically identify close-to-optimal configurations for static workloads, they suffer for dynamic workloads. This happens due to the three fundamental limitations. First, they do not account for performance degradation during the reconfiguration (say due to database restart, which is often needed to apply the new configuration). Second, they do not account for how transient the new workload pattern will be. Third, they overlook the application’s availability requirements during reconfiguration. Our solution, EUNOMIA, addresses all these shortcomings. The fundamental technical contribution is a cost-benefit analyzer that computes the relative cost and the benefit of each reconfiguration action and determines a reconfiguration plan for a future time window. This specifies when to change and to what configurations. We demonstrate its effectiveness for three different workload traces: a multi-tenant, global-scale metagenomics repository (MG-RAST), a bus-tracking workload (Tiramisu), and an HPC data-analytics job queue, all with varying levels of workload complexity and demonstrating dynamic workload changes. We compare the benefit of EUNOMIA in throughput over the default, a static configuration, and a theoretically ideal solution for two widely popular NoSQL databases—Cassandra and Redis.