On the Importance of Opponent Modeling in Auction Markets

The dynamics of financial markets are driven by the interactions between participants, as well as the trading mechanisms and regulatory frameworks that govern these interactions. Decision-makers would rather not ignore the impact of other participants on these dynamics and should employ tools and models that take this into account. To this end, we demonstrate the efficacy of applying opponent-modeling in a number of simulated market settings. While our simulations are simplified representations of actual market dynamics, they provide an idealized "playground" in which our techniques can be demonstrated and tested. We present this work with the aim that our techniques could be refined and, with some effort, scaled up to the full complexity of real-world market scenarios. We hope that the results presented encourage practitioners to adopt opponent-modeling methods and apply them online systems, in order to enable not only reactive but also proactive decisions to be made.

[1]  Michael Mateas,et al.  A data mining approach to strategy prediction , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[2]  Rama Cont,et al.  The Price Impact of Order Book Events , 2010, 1011.6402.

[3]  Barbara Messing,et al.  An Introduction to MultiAgent Systems , 2002, Künstliche Intell..

[4]  Peter Stone,et al.  Autonomous agents modelling other agents: A comprehensive survey and open problems , 2017, Artif. Intell..

[5]  Sandra Carberry,et al.  Techniques for Plan Recognition , 2001, User Modeling and User-Adapted Interaction.

[6]  Sarit Kraus,et al.  Teamwork with Limited Knowledge of Teammates , 2013, AAAI.

[7]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[8]  Manuela M. Veloso,et al.  Planning and Learning by Analogical Reasoning , 1994, Lecture Notes in Computer Science.

[9]  David Carmel,et al.  Opponent Modeling in Multi-Agent Systems , 1995, Adaption and Learning in Multi-Agent Systems.

[10]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Peter A. Beling,et al.  Effects of limit order book information level on market stability metrics , 2014 .

[12]  L. Cosmides,et al.  Evolutionary psychology and the generation of culture, part II: Case study: A computational theory of social exchange , 1989 .

[13]  Michael P. Wellman Trading Agents , 2011, Trading Agents.

[14]  Thore Graepel,et al.  The Mechanics of n-Player Differentiable Games , 2018, ICML.

[15]  Maria Hybinette,et al.  ABIDES: Towards High-Fidelity Market Simulation for AI Research , 2019, ArXiv.

[16]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[17]  Hendrik Bessembinder Price-Time Priority, Order Routing, and Trade Execution Costs in Nyse-Listed Stocks , 2001 .

[18]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[19]  R. Selten,et al.  Game theory and evolutionary biology , 1994 .

[20]  J. Friedman Game theory with applications to economics , 1986 .

[21]  Matthew E. Taylor,et al.  Agent Modeling as Auxiliary Task for Deep Reinforcement Learning , 2019, AIIDE.

[22]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[23]  Tom Schaul,et al.  Reinforcement Learning with Unsupervised Auxiliary Tasks , 2016, ICLR.

[24]  Markus Gsell,et al.  Assessing the Impact of Algorithmic Trading on Markets: A Simulation Approach , 2008, ECIS.

[25]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[26]  David Carmel,et al.  Incorporating Opponent Models into Adversary Search , 1996, AAAI/IAAI, Vol. 1.

[27]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[28]  J. D. Morrow Game Theory for Political Scientists , 1994 .

[29]  Yoav Shoham,et al.  Multiagent Systems - Algorithmic, Game-Theoretic, and Logical Foundations , 2009 .

[30]  M. Avellaneda,et al.  High-frequency trading in a limit order book , 2008 .

[31]  Shimon Whiteson,et al.  Learning with Opponent-Learning Awareness , 2017, AAMAS.

[32]  Ruihong Huang,et al.  On the Dark Side of the Market: Identifying and Analyzing Hidden Order Placements , 2012 .

[33]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[34]  J. Wilder New Concepts in Technical Trading Systems , 1978 .

[35]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[36]  Jacob W. Crandall,et al.  E-HBA: Using Action Policies for Expert Advice and Agent Typification , 2015, AAAI 2015.

[37]  Manuela M. Veloso,et al.  Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork , 1999, Artif. Intell..