Learning Improved Entertainment Trading Strategies for the TAC Travel Game

For almost five years we continually operated a simulation testbed exploring strategies for the TAC Travel game. Building on techniques developed in our recent study of continuous double auctions, we performed an equilibrium analysis of our testbed data, and employed reinforcement learning in the equilibrium environment to derive a new entertainment strategy for this domain. A second iteration of this process led to further improvements. We thus demonstrate that interleaving empirical game-theoretic analysis with reinforcement learning in an effective method for generating stronger trading strategies in this domain.

[1]  Pierre L'Ecuyer,et al.  Efficiency improvement and variance reduction , 1994, Proceedings of Winter Simulation Conference.

[2]  Masahito Yamamoto,et al.  Design of Adaptive Trading Strategy for Trading Agent Competition , 2003 .

[3]  Carlos José Pereira de Lucena,et al.  An Agent Based Architecture for Highly Competitive Electronic Markets , 2005, FLAIRS.

[4]  Peter Stone,et al.  ATTac-2000: an adaptive autonomous bidding agent , 2001, AGENTS '01.

[5]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[6]  Amy Greenwald,et al.  Bid determination in simultaneous actions an agent architecture , 2001, EC '01.

[7]  Victor Naroditskiy,et al.  RoxyBot-06: Stochastic Prediction and Optimization in TAC Travel , 2009, J. Artif. Intell. Res..

[8]  M. P. Wellman,et al.  Price Prediction in a Trading Agent Competition , 2004, J. Artif. Intell. Res..

[9]  Bart Selman,et al.  A principled study of the design tradeoffs for autonomous trading agents , 2003, AAMAS '03.

[10]  Klaus Dorer,et al.  Agent-oriented software engineering for successful TAC participation , 2002, AAMAS '02.

[11]  Pericles A. Mitkas,et al.  A Long-Term Profit Seeking Strategy for Continuous Double Auctions in a Trading Agent Competition , 2006, SETN.

[12]  Michael P. Wellman,et al.  Stronger CDA strategies through empirical game-theoretic analysis and reinforcement learning , 2009, AAMAS.

[13]  Michael P. Wellman,et al.  Strategy exploration in empirical games , 2010, AAMAS.

[14]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[15]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[16]  R. Palmer,et al.  Characterizing effective trading strategies: Insights from a computerized double auction tournament , 1994 .

[17]  Michael P. Wellman,et al.  Empirical game-theoretic analysis of the TAC Supply Chain game , 2007, AAMAS '07.

[18]  David A. McAllester,et al.  Decision-Theoretic Bidding Based on Learned Density Models in Simultaneous, Interacting Auctions , 2003, J. Artif. Intell. Res..

[19]  Michael P. Wellman,et al.  Exploring Large Strategy Spaces in Empirical Game Modeling , 2009 .

[20]  Nicholas R. Jennings,et al.  SouthamptonTAC: An adaptive autonomous trading agent , 2003, TOIT.

[21]  Victor Naroditskiy,et al.  RoxyBot-06: An (SAA)2 TAC Travel Agent , 2007, IJCAI.

[22]  A. Greenwald Bid Determination in Simultaneous Auctions , 2007 .

[23]  Gerald Tesauro,et al.  Strategic sequential bidding in auctions using dynamic programming , 2002, AAMAS '02.

[24]  Michael P. Wellman Methods for Empirical Game-Theoretic Analysis , 2006, AAAI.

[25]  Maria Fasli,et al.  Thalis : A Flexible Trading Agent , 2003 .

[26]  Michael P. Wellman,et al.  Approximate Strategic Reasoning through Hierarchical Reduction of Large Symmetric Games , 2005, AAAI.

[27]  Mats Carlsson,et al.  A trading agent built on constraint programming , 2002 .

[28]  Michael P. Wellman,et al.  Autonomous bidding agents - strategies and lessons from the trading agent competition , 2007 .

[29]  Michael P. Wellman,et al.  Methods for empirical game-theoretic analysis (extended abstract) , 2006 .

[30]  Nicholas R. Jennings,et al.  Strategic bidding in continuous double auctions , 2008, Artif. Intell..

[31]  Michael P. Wellman,et al.  Searching for Walverine 2005 , 2005, AMEC@AAMAS/TADA@IJCAI.