Reinforcement Learning on a Futures Market Simulator

In recent years, it becomes vigorous to forecast a market by using machine learning methods. Since they assume that each trader's individual decisions do not affect market prices at all, most existing works use a past market data set. Meanwhile there is an attempt to analyze economic phenomena by constructing a virtual market simulator, where human and artificial traders really make trades. Since prices in the market are determined by every trader's decisions, it is more realistic and the assumption cannot be applied any more. In this work, we design and evaluate several reinforcement learners on a futures market simulator U-Mart (Unreal Market as an Artificial Research Testbed). After that, we compare our learner to the previous champions of U-Mart competitions.

[1]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[2]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[3]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[4]  Hiroshi Sato,et al.  U-Mart: A Test Bed for Interdisciplinary Research into Agent-Based Artificial Markets , 2001 .

[5]  Michael Kearns,et al.  Reinforcement learning for optimized trade execution , 2006, ICML.

[6]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[7]  Hisao Ishibuchi,et al.  Behavior Analysis of Futures Trading Agents Using Fuzzy Rule Extraction , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[8]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[9]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .