The BigKClustering approach for document clustering using Hadoop MapReduce

Clustering is an efficient data mining as well as machine-learning method when we need to get an insight of the objects of a dataset that could be grouped together. K-Means is one of the most commonly used methods of clustering, due to its high quality results and low time cost. However, using the K-Means algorithm in document clustering over large-scale collections can lead to unpredictable time costs, since duration of a K-Means iteration tends to grow as the number of iterations grows. In this paper we first present some of the most promising alternatives for document clustering over such 'big data' (large-scale) collections. We also present our variation of an existing K-Means-based algorithm, known as BigKClustering (BKC) so that it can be applied in document clustering. The proposed adjustment of BKC is then implemented using Hadoop MapReduce to handle big (text) data collections efficiently and experimentally tested over a real cluster environment. As it comes out of the experiments, it leads to acceptable clustering quality as well as significant execution time improvements (compared to K-Means), thus constituting a promising clustering approach for big document collections.