Revisiting the Role of Pipelined Parallelism in Multi-Join Query Processing

Multi-join queries are the core of any integration service that integrates data from multiple distributed data sources. Due to the large number of data sources and possibly high volumes of data, the evaluation of multi-join queries faces increasing scalability concerns. Parallel processing has been applied to tackle this problem. State-of-the-art parallel multi-join query processing commonly assume that the application of maximal pipelined parallelism leads to superior performance. In this paper, we instead illustrate that this assumption does not generally hold. We investigate how best to combine pipelined parallelism with alternate forms of parallelism to achieve an overall effective parallel processing strategy. An m-way bushy parallel processing strategy is proposed. Experimental studies are conducted on an actual software system over a cluster of high-performance PCs. The experimental results confirm that the proposed parallel processing strategy leads to an on average of 50% improvement in terms of total processing time in comparison to existing state-of-the-art solutions.