Exploiting Bloom Filters for Efficient Joins in MapReduce

MapReduce is a programming model that is extensively used for largescale data analysis. However, it is inefficient to perform join operations using MapReduce, because large intermediate results are produced, even in cases where only a small fraction of input data participate in the join. We alleviate this problem by exploiting Bloom filters within a single MapReduce job. We create Bloom filters for an input dataset, and filter out the redundant records in the other input dataset in the map phase. To do this, we modify the MapReduce framework in two ways. First, map tasks are scheduled according to the processing order of input datasets. Second, Bloom filters are dynamically created in a distributed fashion. We propose two map task scheduling policies and provide a method to determine the processing order based on the estimated cost. Our experimental results show that the proposed techniques decrease the size of intermediate results and can improve the execution time.