A Reinforcement-Learning-Based Evolutionary Algorithm Using Solution Space Clustering For Multimodal Optimization Problems

In evolutionary algorithms, how to effectively select interactive solutions for generating offspring is a challenging problem. Though many operators are proposed, most of them select interactive solutions (parents) randomly, having no specificity for the features of landscapes in various problems. To address this issue, this paper proposes a reinforcement-learning-based evolutionary algorithm to select solutions within the approximated basin of attraction. In the algorithm, the solution space is partitioned by the k-dimensional tree, and features of subspaces are approximated with respect to two aspects: objective values and uncertainties. Accordingly, two reinforcement learning (RL) systems are constructed to determine where to search: the objective-based RL exploits basins of attraction (clustered subspaces) and the uncertainty-based RL explores subspaces that have been searched comparatively less. Experiments are conducted on widely used benchmark functions, demonstrating that the algorithm outperforms three other popular multimodal optimization algorithms.