Quantifying Long-Range Interactions and Coherent Structure in Multi-Agent Dynamics

We develop and apply several novel methods quantifying dynamic multi-agent team interactions. These interactions are detected information-theoretically and captured in two ways: via (i) directed networks (interaction diagrams) representing significant coupled dynamics between pairs of agents, and (ii) state-space plots (coherence diagrams) showing coherent structures in Shannon information dynamics. This model-free analysis relates, on the one hand, the information transfer to responsiveness of the agents and the team, and, on the other hand, the information storage within the team to the team's rigidity and lack of tactical flexibility. The resultant interaction and coherence diagrams reveal implicit interactions, across teams, that may be spatially long-range. The analysis was verified with a statistically significant number of experiments (using simulated football games, produced during RoboCup 2D Simulation League matches), identifying the zones of the most intense competition, the extent and types of interactions, and the correlation between the strength of specific interactions and the results of the matches.

[1]  I. Couzin,et al.  Effective leadership and decision-making in animal groups on the move , 2005, Nature.

[2]  Karl J. Friston.  Functional and effective connectivity in neuroimaging: A synthesis , 1994 .

[3]  Keith Davids,et al.  Science of winning soccer: Emergent pattern-forming dynamics in association football , 2013, Journal of Systems Science and Complexity.

[4]  Keiko Yokoyama,et al.  Emergence of self-similarity in football dynamics , 2014 .

[5]  O. Sporns.  Networks of the Brain , 2010 .

[6]  Yukio-Pegio Gunji,et al.  Information transfer in a swarm of soldier crabs , 2016, Artificial Life and Robotics.

[7]  Hiroaki Kitano,et al.  RoboCup: Today and Tomorrow - What we have learned , 1999, Artif. Intell..

[8]  Luís Paulo Reis,et al.  Performance analysis in soccer: a Cartesian coordinates based approach using RoboCup data , 2011, Soft Computing.

[9]  Risto Miikkulainen,et al.  Evolving Keepaway Soccer Players through Task Decomposition , 2003, GECCO.

[10]  Zack J. Butler,et al.  Million Module March: Scalable Locomotion for Large Self-Reconfiguring Robots , 2008, Int. J. Robotics Res..

[11]  Serge Kernbach,et al.  Reconfigurable modular robotics , 2014, Robotics Auton. Syst..

[12]  Nuttapong Chentanez,et al.  Intrinsically Motivated Reinforcement Learning , 2004, NIPS.

[13]  Mikhail Prokopenko,et al.  Measuring Information Dynamics in Swarms , 2014 .

[14]  H. Touchette,et al.  A network theory analysis of football strategies , 2012, 1206.6904.

[15]  Oliver Obst,et al.  Gliders2014: Dynamic Tactics with Voronoi Diagrams , 2014 .

[16]  Wenguo Liu,et al.  Modeling and Optimization of Adaptive Foraging in Swarm Robotic Systems , 2010, Int. J. Robotics Res..

[17]  Joseph T. Lizier,et al.  Measuring the Dynamics of Information Processing on a Local Scale in Time and Space , 2014 .

[18]  Keyan Zahedi,et al.  Linear combination of one-step predictive information with an external reward in an episodic policy gradient setting: a critical analysis , 2013, Front. Psychol..

[19]  Jochen Kaiser,et al.  Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. , 2011, Progress in biophysics and molecular biology.

[20]  Albert Y. Zomaya,et al.  Information modification and particle collisions in distributed computation. , 2010, Chaos.

[21]  Mikhail Prokopenko,et al.  Flexible Synchronisation within RoboCup Environment: A Comparative Analysis , 2000, RoboCup.

[22]  Hiroaki Kitano,et al.  The RoboCup Synthetic Agent Challenge 97 , 1997, IJCAI.

[23]  J. Duch,et al.  Quantifying the Performance of Individual Players in a Team Activity , 2010, PloS one.

[24]  X. R. Wang,et al.  Quantifying and Tracing Information Cascades in Swarms , 2012, PloS one.

[25]  Mikhail Prokopenko,et al.  Evolving Spatiotemporal Coordination in a Modular Robotic System , 2006, SAB.

[26]  Joseph T. Lizier,et al.  JIDT: An Information-Theoretic Toolkit for Studying the Dynamics of Complex Systems , 2014, Front. Robot. AI.

[27]  Mikhail Prokopenko,et al.  An Information Criterion for Inferring Coupling of Distributed Dynamical Systems , 2016, Front. Robot. AI.

[28]  Nikos A. Vlassis,et al.  Non-communicative multi-robot coordination in dynamic environments , 2005, Robotics Auton. Syst..

[29]  Oliver Obst,et al.  Gliders 2016 : Integrating multi-agent approaches to tactical diversity , 2016 .

[30]  Luís Paulo Reis,et al.  Multi-robot coordination using Setplays in the middle-size and simulation leagues , 2011, Mechatronics.

[31]  Mikhail Prokopenko,et al.  Relating the Entropy of Joint Beliefs to Multi-agent Coordination , 2002, RoboCup.

[32]  Hiroaki Kitano,et al.  The RoboCup humanoid challenge as the millennium challenge for advanced robotics , 1998, Adv. Robotics.

[33]  Joseph T. Lizier,et al.  Directed Information Measures in Neuroscience , 2014 .

[34]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[35]  Eckehard Olbrich,et al.  Autonomy: An information theoretic perspective , 2008, Biosyst..

[36]  J. Crutchfield,et al.  Regularities unseen, randomness observed: levels of entropy convergence. , 2001, Chaos.

[37]  Oliver Obst,et al.  Gliders2012: Tactics with Action-dependent Evaluation Functions , 2012 .

[38]  Albert Y. Zomaya,et al.  Local information transfer as a spatiotemporal filter for complex systems. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[40]  L. Faes,et al.  Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  Carl S. McTague,et al.  The organization of intrinsic computation: complexity-entropy diagrams and the diversity of natural information processing. , 2008, Chaos.

[42]  Oliver Obst,et al.  Gliders2013: Tactical Analysis with Information Dynamics , 2013 .

[43]  Peter Stone,et al.  The RoboCup Soccer Server and CMUnited Clients: Implemented Infrastructure for MAS Research , 2003, Autonomous Agents and Multi-Agent Systems.

[44]  Theo Geisel,et al.  Model-Free Reconstruction of Excitatory Neuronal Connectivity from Calcium Imaging Signals , 2012, PLoS Comput. Biol..

[45]  Jakob Heinzle,et al.  Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity , 2010, Journal of Computational Neuroscience.

[46]  Manuela M. Veloso,et al.  Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork , 1999, Artif. Intell..

[47]  M. Prokopenko,et al.  Evolving Spatiotemporal Coordination in a Modular Robotic System , 2006, SAB.

[48]  Zhe Xu,et al.  Decentralized Coordinated Tracking with Mixed Discrete–Continuous Decisions , 2013, J. Field Robotics.

[49]  Albert Y. Zomaya,et al.  Local measures of information storage in complex distributed computation , 2012, Inf. Sci..

[50]  Gordon Pipa,et al.  Assessing coupling dynamics from an ensemble of time series , 2010, Entropy.

[51]  Raul Vicente,et al.  Transfer Entropy in Neuroscience , 2014 .

[52]  Xiaofeng Han,et al.  Autonomous navigation of wireless robot swarms with covert leaders , 2007 .

[53]  Oliver Obst,et al.  Towards Quantifying Interaction Networks in a Football Match , 2013, RoboCup.

[54]  Mikhail Prokopenko,et al.  Evaluating Team Performance at the Edge of Chaos , 2003, RoboCup.

[55]  Risto Miikkulainen,et al.  Evolving Soccer Keepaway Players Through Task Decomposition , 2005, Machine Learning.

[56]  Robert Fitch,et al.  Online decentralized information gathering with spatial–temporal constraints , 2014, Auton. Robots.

[57]  Raul Vicente,et al.  Efficient Estimation of Information Transfer , 2014 .

[58]  Lenka Pitonakova,et al.  Information flow principles for plasticity in foraging robot swarms , 2016, Swarm Intelligence.

[59]  Joseph T. Lizier,et al.  Multivariate construction of effective computational networks from observational data , 2012 .

[60]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[61]  Albert Y. Zomaya,et al.  Coherent information structure in complex computation , 2012, Theory in Biosciences.

[62]  Daniele Marinazzo,et al.  Causal Information Approach to Partial Conditioning in Multivariate Data Sets , 2011, Comput. Math. Methods Medicine.

[63]  Albert Y. Zomaya,et al.  The local information dynamics of distributed computation in complex systems , 2012 .

[64]  Dieter Armbruster,et al.  Basketball Teams as Strategic Networks , 2012, PloS one.

[65]  Hugh F. Durrant-Whyte,et al.  Decentralised decision making in heterogeneous teams using anonymous optimisation , 2009, Robotics Auton. Syst..

[66]  Mahsa Asadi,et al.  Cyrus 2D Simulation Team Description Paper2018 , 2020, ArXiv.

[67]  Gordon Pipa,et al.  Transfer entropy—a model-free measure of effective connectivity for the neurosciences , 2010, Journal of Computational Neuroscience.

[68]  Martin A. Riedmiller,et al.  Reinforcement learning for robot soccer , 2009, Auton. Robots.

[69]  Yu Sun,et al.  Information Transfer in Swarms with Leaders , 2014, ArXiv.

[70]  Luca Faes,et al.  Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series , 2012, Comput. Biol. Medicine.

[71]  Robert Fitch,et al.  Communication-aware information gathering with dynamic information flow , 2015, Int. J. Robotics Res..

[72]  Yu Sun,et al.  Modeling and Analyzing Large Swarms with Covert Leaders , 2013, 2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems.

[73]  W. Bialek,et al.  Statistical mechanics for natural flocks of birds , 2011, Proceedings of the National Academy of Sciences.

[74]  Sachit Butail,et al.  Model-free information-theoretic approach to infer leadership in pairs of zebrafish. , 2016, Physical review. E.