Multi-core vs. I/O Wall: The Approaches to Conquer and Cooperate

Multi-core comes to be the mainstream of processor techniques. The data-intensive OLAP relies on inexpensive disks as massive data storage device, so the enhanced processing power oppose to I/O bottleneck in big data OLAP applications becomes more critical because the latency gap between I/O and multi-core gets even larger. In this paper, we focus on the disk resident OLAP with large dataset, exploiting the power of multi-core processing under I/O bottleneck. We propose optimizations for schema-aware storage layout, parallel accessing and I/O latency aware concurrent processing. On the one hand I/O bottleneck should be conquered to reduce latency for multi-core processing, on the other hand we can make good use of I/O latency for heavy concurrent query workload with multi-core power. We design experiments to exploit parallel and concurrent processing power for multi-core with DDTA-OLAP engine which minimizes the star-join cost by directly dimension tuple accessing technique. The experimental results show that we can achieve maximal speedup ratio of 103 for multi-core concurrent query processing in DRDB scenario.