Empowerment and State-dependent Noise - An Intrinsic Motivation for Avoiding Unpredictable Agents

Empowerment is a recently introduced intrinsic motivation algorithm based on the embodiment of an agent and the dynamics of the world the agent is situated in. Computed as the channel capacity from an agent’s actuators to an agent’s sensors, it offers a quantitative measure of how much an agent is in control of the world it can perceive. In this paper, we expand the approximation of empowerment as a Gaussian linear channel to compute empowerment based on the covariance matrix between actuators and sensors, incorporating statedependent noise. This allows for the first time the study of continuous systems with several agents. We found that if the behaviour of another agent cannot be predicted accurately, then interacting with that agent will decrease the empowerment of the original agent. This leads to behaviour realizing collision avoidance with other agents, purely from maximising an agent’s empowerment.

[1]  J. Uexküll.  Umwelt und Innenwelt der Tiere , 1921 .

[2]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[3]  M. Csíkszentmihályi.  Beyond boredom and anxiety , 1975 .

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[6]  Karl Sims,et al.  Evolving 3d morphology and behavior by competition , 1994 .

[7]  Ralf Der,et al.  Homeokinesis - A new principle to back up evolution with learning , 1999 .

[8]  Emre Telatar,et al.  Capacity of Multi-antenna Gaussian Channels , 1999, Eur. Trans. Telecommun..

[9]  Touchette,et al.  Information-theoretic limits of control , 1999, Physical review letters.

[10]  A. U.S.,et al.  Predictability , Complexity , and Learning , 2002 .

[11]  Luc Steels,et al.  The Autotelic Principle , 2003, Embodied Artificial Intelligence.

[12]  Chrystopher L. Nehaniv,et al.  Empowerment: a universal agent-centric measure of control , 2005, 2005 IEEE Congress on Evolutionary Computation.

[13]  Chrystopher L. Nehaniv,et al.  All Else Being Equal Be Empowered , 2005, ECAL.

[14]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[15]  M. Prokopenko,et al.  Evolving Spatiotemporal Coordination in a Modular Robotic System , 2006, SAB.

[16]  Olaf Sporns,et al.  Methods for quantifying the informational structure of sensory and motor data , 2007, Neuroinformatics.

[17]  Philippe Capdepuy,et al.  Maximization of Potential Information Flow as a Universal Utility for Collective Behaviour , 2007, 2007 IEEE Symposium on Artificial Life.

[18]  Eckehard Olbrich,et al.  Autonomy: An information theoretic perspective , 2008, Biosyst..

[19]  Chrystopher L. Nehaniv,et al.  Keep Your Options Open: An Information-Based Driving Principle for Sensorimotor Systems , 2008, PloS one.

[20]  Ralf Der,et al.  Predictive information and explorative behavior of autonomous robots , 2008 .

[21]  L. Munari.  How the body shapes the way we think — a new view of intelligence , 2009 .

[22]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[23]  S. West,et al.  Evolutionary Theory and the Ultimate–Proximate Distinction in the Human Behavioral Sciences , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[24]  Philippe Capdepuy,et al.  Perception–action loops of multiple agents: informational aspects and the impact of coordination , 2011, Theory in Biosciences.

[25]  Peter Stone,et al.  Empowerment for continuous agent—environment systems , 2011, Adapt. Behav..

[26]  Christoph Salge,et al.  Approximation of Empowerment in the continuous Domain , 2013, Adv. Complex Syst..

[27]  H. Vincent Poor,et al.  NOMA: An Information Theoretic Perspective , 2015, ArXiv.