Tracking extrema in dynamic environments using Probability Collectives Multi-agent Systems

We present a study of extrema-tracking in dynamic environments using Probability Collectives Multi-agent Systems (PCMAS). In contrast to traditional biologically-inspired algorithms, Probability-Collectives (PC) based methods do not update populations of solutions; instead, they update an explicitly parameterized probability distribution over the space of solutions. Three versions of PCMAS in the extrema-tracking tasks are investigated: PCMAS1 (original PCMAS settings), PCMAS2 (temperature T — a factor controlling the balance between exploration and exploitation of the search space — is reset to the initial state when an environmental change takes place), as well as PCMAS3 (in addition to T being reset to the initial state, the probability distributions are also reset to uniform when an environment changes). By allowing PCMAS to detect changes in environments to re-explore the search space, we show that PCMAS2 and PCMAS3 significantly outperform the original PCMAS (i.e., PCMAS1). The study of the PCMAS in changing environments therefore sheds light on how this multi-agent methodology advances the current state of research in agent-based models for dynamic optimization problems.