Evolutionary multi-objective automatic clustering enhanced with quality metrics and ensemble strategy

Abstract Automatic clustering problem, which needs to detect the appropriate clustering without a pre-defined number of clusters (k), is difficult and challenging in unsupervised learning owing to the lack of prior domain knowledge. Despite a rising tendency with the application of evolutionary multi-objective optimization (EMO) techniques for automatic clustering, there still exist some obvious under-explored issues. In this paper, we resort to quality metrics and ensemble strategy for the sake of explicit/implicit knowledge discovery to guide the optimization process. The quality and diversity of solutions defined in terms of cluster validities, as similar to performance indicator for multi-objective optimization, are applied to assist in addressing automatic clustering problems and decreasing unnecessary computational overhead. To be specific, the main components like initialization, reproduction operations, and environmental selection which involved during EMO based automatic clustering are discussed and refined. For the determination of the final partitioning, quality metrics and cluster ensemble strategy are both considered to improve the retrieve system in the unsupervised way. Experiments are conducted from several different aspects and the corresponding analyses are provided, which confirm that the proposals are more efficient and effective for automatic clustering.