Multidimensional Preference Query Optimization on Infrastructure Monitoring Systems

Performance monitoring systems collect and analyze data, and provide insight on availability, performance, and capacity from various monitored targets. However, the analytics in online performance monitoring systems have often been limited due to the query performance of large-scale multidimensional data. The existing approaches for preference or ranking queries generally include approaches with and without pre-processing. The approaches without pre-processing work well with small data sets. For larger datasets, the preference query is usually slow or not time feasible. In this paper, we introduce and evaluate an optimized query approach for performance monitoring systems using the bit-sliced index (BSI). The objective is to enable users to perform fast interactive queries with various querying criteria on multiple dimensions. The experiments cover preference top-k queries on the proposed approach with BSI, an approach using bitmap indexing for top-k queries, and the sequential scan sort top-k selection algorithms. The evaluation covers the single attribute query, multiple attributes weighted sum query, and multidimensional grouping using a real-time performance monitoring system data with several hundred thousand records. Working with a single attribute top-k evaluation, the proposed bit-sliced approach on average outperforms the bitmap approach and the sequential scan approach by a factor of 2 and a factor of 100+ respectively. Similarly, with multiple attributes sum top-k evaluation, it is 5 times faster than the sequential scan approach as well.