A Spark-based differential evolution with grouping topology model for large-scale global optimization

Over the past few years, cloud computing model (e.g., Spark) has aroused huge attention. Differential evolution (DE) has been applied to cloud computing models by a number of researchers for its merits in solving large-scale global optimization problems (LSGO), and remarkable results have been achieved. Moreover, we noticed that a combination of better topology and migration strategy is critical to solve the mentioned problems when DE algorithm acts as an internal optimizer for Spark cloud computing model. However, rare studies have been conducted to combine the combination to enhance the performance of DE algorithm for solving large-scale global optimization problems. Thus, inspired by the mentioned insights, we propose a novel grouping topology model that uses DE variants as internal optimizers to solve LSGO problems, called SgtDE. In SgtDE, population is split into subgroups evenly, and various topology structures are introduced to migrate individuals between and within subgroups. In this paper, five types of DE are adopted as the internal optimizers. By comparing the 20 benchmark functions presented on CEC2010, the results demonstrate that the SgtDE, especially a combination of better topology and migration strategy, exhibits significant performance in applying various DE variants. Thus, the SgtDE can act as the next generation optimizer of the cloud computing platform.