Highly scalable Erlang framework for agent-based metaheuristic computing

Abstract Difficult search and optimization problems, usually solved by metaheuristics, are very often implemented in concurrent and parallel environment, as many metaheuristics (e.g. population- or agent-based) are inherently easy to parallelize. Therefore search for easy-to-use, robust and efficient frameworks dedicated for such computing methods, especially in the era of ubiquitous many and multi-core systems, is very desirable. Indeed, the development of multi-core architectures is incredibly fast and multicore CPUs can be found nowadays not only in supercomputers, but also in ordinary laptops or even phones. Efficient use of multicore architectures requires applying suitable languages and technologies, like Erlang. Its concurrency model, based on lightweight processes and asynchronous message-passing, seems very well suited for running massively concurrent code on many cores. Most of existing Erlang industrial applications are deployed on computers with up to 24 CPU cores, and there are hardly any reports on using Erlang on architectures exceeding 32 physical cores. In this paper we present our experiences with developing a concurrent Erlang-based computing platform, scaling computationally-intensive and memory-intensive applications up to 64 cores, using as examples global optimization and urban traffic planning problems.