ME2: A Scalable Modular Meta-heuristic for Multi-modal Multi-dimension Optimization

Map, Explore & Exploit (ME2) is a scalable meta-heuristic for problems in the field of multi-modal, multi-dimension optimization. It has a modular design with three phases, as reflected by its name. Its first phase (Map) generates a set of samples that is mostly uniformly distributed over the search space. The second phase (Explore) explores the neighbourhood of each sample point using an evolutionary strategy, to find a good - not necessarily optimal - set of neighbours. The third phase (Exploit) optimizes the results of the second phase. This final phase applies a simple gradient descent algorithm to find the local optima for each and all of the neighbourhoods, with the objective of finding a/the global optima of the whole space. The performance of ME2 is compared, on a fair basis, with the performance of benchmark optimization algorithms: Genetic Algorithms, Particle Swarm Optimization, Simulated Annealing and Covariance Matrix Adaptation Evolution Strategy. In most test cases it finds the global optima earlier than the other algorithms. It also scales-up, without loss of performance, to higher dimensions. Due to the distributed nature of ME2’s second and third phase, it can be comprehensively parallelized. The search & optimization process during these two phases can be applied to each sample point independently of all the others. A multi-threaded version of ME2 was written and compared to its single-threaded version, resulting in a near-linear speed-up as a function of the number of cores employed.