Learning to classify and imitate trading agents in continuous double auction markets

Continuous double auctions such as the limit order book employed by exchanges are widely used in practice to match buyers and sellers of a variety of financial instruments. In this work, we develop an agent-based model for trading in a limit order book and show (1) how opponent modelling techniques can be applied to classify trading agent archetypes and (2) how behavioural cloning can be used to imitate these agents in a simulated setting. We experimentally compare a number of techniques for both tasks and evaluate their applicability and use in real-world scenarios.

[1]  Sarit Kraus,et al.  Teamwork with Limited Knowledge of Teammates , 2013, AAAI.

[2]  Sandra Carberry,et al.  Techniques for Plan Recognition , 2001, User Modeling and User-Adapted Interaction.

[3]  Thore Graepel,et al.  The Mechanics of n-Player Differentiable Games , 2018, ICML.

[4]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .

[5]  Michael P. Wellman,et al.  Welfare Effects of Market Making in Continuous Double Auctions , 2015, AAMAS.

[6]  Peter Stone,et al.  Autonomous agents modelling other agents: A comprehensive survey and open problems , 2017, Artif. Intell..

[7]  Fei-Fei Li,et al.  Deep visual-semantic alignments for generating image descriptions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ruihong Huang,et al.  On the Dark Side of the Market: Identifying and Analyzing Hidden Order Placements , 2012 .

[9]  Jacob W. Crandall,et al.  E-HBA: Using Action Policies for Expert Advice and Agent Typification , 2015, AAAI 2015.

[10]  David Carmel,et al.  Opponent Modeling in Multi-Agent Systems , 1995, Adaption and Learning in Multi-Agent Systems.

[11]  David D. Cox,et al.  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures , 2013, ICML.

[12]  A. Kyle.  Continuous Auctions and Insider Trading , 1985 .

[13]  Pieter Abbeel,et al.  An Algorithmic Perspective on Imitation Learning , 2018, Found. Trends Robotics.

[14]  M. Dacorogna,et al.  A geographical model for the daily and weekly seasonal volatility in the foreign exchange market , 1993 .

[15]  Trevor Hastie,et al.  Multi-class AdaBoost ∗ , 2009 .

[16]  Michael P. Wellman,et al.  Spoofing the Limit Order Book: An Agent-Based Model , 2017, AAMAS.

[17]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[18]  David Vandyke,et al.  Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems , 2015, EMNLP.

[19]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[20]  Maria Hybinette,et al.  ABIDES: Towards High-Fidelity Market Simulation for AI Research , 2019, ArXiv.

[21]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[22]  J. Bouchaud,et al.  Trades, Quotes and Prices: Financial Markets Under the Microscope , 2018 .

[23]  David Carmel,et al.  Incorporating Opponent Models into Adversary Search , 1996, AAAI/IAAI, Vol. 1.

[24]  Matthew E. Taylor,et al.  Agent Modeling as Auxiliary Task for Deep Reinforcement Learning , 2019, AIIDE.

[25]  Dhananjay K. Gode,et al.  Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality , 1993, Journal of Political Economy.

[26]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[27]  Tom Schaul,et al.  Reinforcement Learning with Unsupervised Auxiliary Tasks , 2016, ICLR.

[28]  Yoav Shoham,et al.  Multiagent Systems - Algorithmic, Game-Theoretic, and Logical Foundations , 2009 .

[29]  Manuela Veloso,et al.  On the Importance of Opponent Modeling in Auction Markets , 2019, 1911.12816.

[30]  Manuela M. Veloso,et al.  Planning and Learning by Analogical Reasoning , 1994, Lecture Notes in Computer Science.

[31]  Manuela M. Veloso,et al.  Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork , 1999, Artif. Intell..

[32]  Tanmoy Chakraborty,et al.  Market making and mean reversion , 2011, EC '11.

[33]  Michael Mateas,et al.  A data mining approach to strategy prediction , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[34]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[35]  Markus Gsell,et al.  Assessing the Impact of Algorithmic Trading on Markets: A Simulation Approach , 2008, ECIS.

[36]  Michael P. Wellman.  Trading Agents , 2011, Trading Agents.

[37]  Barbara Messing,et al.  An Introduction to MultiAgent Systems , 2002, Künstliche Intell..

[38]  Peter A. Beling,et al.  Effects of limit order book information level on market stability metrics , 2014 .

[39]  Shimon Whiteson,et al.  Learning with Opponent-Learning Awareness , 2017, AAMAS.

[40]  Steve Y. Yang,et al.  Gaussian process-based algorithmic trading strategy identification , 2012 .

[41]  Tobias Preis,et al.  Price-Time Priority and Pro Rata Matching in an Order Book Model of Financial Markets , 2011 .