Clustering Approach for Data Lake Based on Medoid’s Ranking Strategy

A number of conventional clustering algorithms suffer from poor scalability, especially for data lake. Thus many modified clustering algorithms have been proposed to speed up these conventional algorithms based on the employment of data sampling techniques. However, these representations require the number of clusters to proceed to centroid selection for final data clustering. To address this limitation, this paper develops a two-phase clustering-based methodology. In the first phase, rather than attempting to construct a random sampling, we define a novel approach that computes plausible sample points, uses them as centroids for the final clusters. To speedup our clustering algorithm in the second phase we propose a parallelization scheme in conjunction with a Spark parallel processing implementation. Computational experiments reveal that the Global sampling method is more effective in terms of both quality and stability compared to the most popular K-means algorithm for the same parameter settings.