Emerging social awareness: Exploring intrinsic motivation in multiagent learning

Recently, a novel framework has been proposed for intrinsically motivated reinforcement learning (IMRL) in which a learning agent is driven by rewards that include not only information about what the agent must accomplish in order to “survive”, but also additional reward signals that drive the agent to engage in other activities, such as playing or exploring, because they are “inherently enjoyable”. In this paper, we investigate the impact of intrinsic motivation mechanisms in multiagent learning scenarios, by considering how such motivational system may drive an agent to engage in behaviors that are “socially aware”. We show that, using this approach, it is possible for agents to learn individually to acquire socially aware behaviors that tradeoff individual well-fare for social acknowledgment, leading to a more successful performance of the population as a whole.

[1]  Andrew G. Barto,et al.  An intrinsic reward mechanism for efficient exploration , 2006, ICML.

[2]  Andrew G. Barto,et al.  An Adaptive Robot Motivational System , 2006, SAB.

[3]  Richard L. Lewis,et al.  Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective , 2010, IEEE Transactions on Autonomous Mental Development.

[4]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination , 2004 .

[5]  Pierre-Yves Oudeyer,et al.  Intrinsically Motivated Machines , 2006, 50 Years of Artificial Intelligence.

[6]  Richard L. Lewis,et al.  Where Do Rewards Come From , 2009 .

[7]  Pierre-Yves Oudeyer,et al.  Motivational principles for visual know-how development , 2003 .

[8]  Ann Nowé,et al.  Evolutionary game theory and multi-agent reinforcement learning , 2005, The Knowledge Engineering Review.

[9]  T. Bergstrom.  On the Evolution of Altruistic Ethical Rules for Siblings , 1995 .

[10]  Andrew Y. Ng,et al.  Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping , 1999, ICML.

[11]  D. E. Matthews.  Evolution and the Theory of Games , 1977 .

[12]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[13]  Richard L. Lewis,et al.  Internal Rewards Mitigate Agent Boundedness , 2010, ICML.

[14]  G. Baldassarre,et al.  Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot , 2007, 2007 IEEE 6th International Conference on Development and Learning.

[15]  A. Barto,et al.  Intrinsic Motivation For Reinforcement Learning Systems , 2005 .

[16]  Jürgen Schmidhuber,et al.  Formal Theory of Fun and Creativity , 2010, ECML/PKDD.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[19]  Lee Spector,et al.  Genetic Programming for Reward Function Search , 2010, IEEE Transactions on Autonomous Mental Development.

[20]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[21]  J. Bach,et al.  Principles of Synthetic Intelligence: Psi: An Architecture of Motivated Cognition , 2009 .

[22]  Andrew W. Moore,et al.  Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time , 1993, Machine Learning.

[23]  Andrew G. Barto,et al.  Intrinsically Motivated Reinforcement Learning: A Promising Framework for Developmental Robot Learning , 2005 .

[24]  F. D. de Waal.  Putting the altruism back into altruism: the evolution of empathy. , 2008, Annual review of psychology.