Benchmarking Collective Perception: New Task Difficulty Metrics for Collective Decision-Making

This paper presents nine different visual patterns for a Collective Perception scenario as new benchmark problems, which can be used for the future development of more efficient collective decision-making strategies. The experiments using isomorphism and three of the well-studied collective decision-making mechanisms are conducted to validate the performance of the new scenarios. The results on a diverse set of problems show that the real task difficulty lies not only in the quantity ratio of the features in the environment but also in their distributions and the clustering levels. Given this, two new metrics for the difficulty of the task are additionally proposed and evaluated on the provided set of benchmarks.