Competitive Benchmarking: Lessons Learned from the Trading Agent Competition

Many important developments in artificial intelligence have been stimulated by organized competitions that tackle interesting, difficult challenge problems, such as chess, robot soccer, poker, robot navigation, stock trading, and others. Economics and artificial intelligence share a strong focus on rational behavior. Yet the real-time demands of many domains do not lend hemselves to traditional assumptions of rationality. This is the case in many trading environments, where self-interested entities need to operate subject to limited time and information. With the web mediating an ever broader range of transactions and opening the door for participants to concurrently trade across multiple markets, there is a growing need for technologies that empower participants to rapidly evaluate very large numbers of alternatives in the face of constantly changing market conditions. AI and machine-learning techniques, including neural networks and genetic algorithms, are already routinely used in support of automated trading scenarios. Yet, the deployment of these technologies remains limited, and their proprietary nature precludes the type of open benchmarking that is critical for further scientific progress.