Complex Dynamics of Single Agent Choice Governed by Dual-Channel Multi-Mode Reinforcement Learning

According to the modern theory of adaption of socioeconomic systems to unknown environments only the interaction between agents can be responsible for various emergent phenomena governed by decision-making and agent learning. Previously we advocated the idea that adopting a more complex model for the agent individual behavior including rational and irrational reasons for decision-making, a more diverse spectrum of macro-level behaviors can be expected. To justify this idea we have developed a model based on the reinforcement learning paradigm extended to including an additional channel of processing information; an agent is biased by novelty seeking, the intrinsic inclination for exploration. In the present paper we demonstrate that the behavior of the single novelty-seeking agent may be extremely irregular and the concepts of chaos can be used to characterize it.

[1]  Ihor Lubashevsky,et al.  Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics , 2009, 0911.2406.

[2]  Eizo Akiyama,et al.  Chaos in learning a simple two-person game , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[3]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[4]  John L. Crompton,et al.  Measuring novelty seeking in tourism. , 1992 .

[5]  David S. Leslie,et al.  Individual Q-Learning in Normal Form Games , 2005, SIAM J. Control. Optim..

[6]  Joan Y. Chiao,et al.  Genetic Determinants of Financial Risk Taking , 2009, PloS one.

[7]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[8]  Aram Galstyan,et al.  Dynamics of Boltzmann Q learning in two-player two-action games. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[9]  T. Ordoña,et al.  Feeling Good: The Science of Well-Being , 2005 .

[10]  J. Crutchfield,et al.  Coupled replicator equations for the dynamics of learning in multiagent systems. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  Ihor Lubashevsky,et al.  Unstable Dynamics of Adaptation in unknown Environment due to Novelty seeking , 2014, Adv. Complex Syst..