Research on a New Density Clustering Algorithm Based on MapReduce

The empirical solution parameters for the Density-Based Spatial Clustering of Applications with Noise(DBSCAN) resulted in poor Clustering effect and low execution efficiency, An adaptive DBSCAN algorithm based on genetic algorithm and MapReduce programming framework is proposed. The genetic algorithm (minPts) and scanning radius size (Eps) optimized intensive interval threshold, at the same time, combined with the similarities and differences of data sets using the Hadoop cluster parallel computing ability of two specifications, the data is reasonable of serialization, finally realizes the adaptive parallel clustering efficiently. Experimental results show that the improved algorithm (GA) - DBSCANMR when dealing with the data set of magnitude 3 times execution efficiency is improved DBSCAN algorithm, clustering quality improved by 10%, and this trend increases as the amount of data, provides a more precise threshold DBSCAN algorithm to determine the implementation of the method.

[1]  Yi-Leh Wu,et al.  Adaptive density-based spatial clustering of applications with noise (DBSCAN) according to data , 2015, 2015 International Conference on Machine Learning and Cybernetics (ICMLC).

[2]  Ling Tian,et al.  A Parallel DBSCAN Algorithm Based on Spark , 2016, 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom).

[3]  Lingjuan Li,et al.  Research on Clustering Algorithm and Its Parallelization Strategy , 2011, 2011 International Conference on Computational and Information Sciences.

[4]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[5]  Dilip B. Kotak,et al.  GRIDBSCAN: GRId Density-Based Spatial Clustering of Applications with Noise , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[6]  Di Ma,et al.  MR-DBSCAN: An Efficient Parallel Density-Based Clustering Algorithm Using MapReduce , 2011, 2011 IEEE 17th International Conference on Parallel and Distributed Systems.