The field of reinforcement learning aims to develop algorithms that use experience to optimize behavior in sequential decision problems such as (partially observable) Markov decision processes ((PO)MDPs). In such problems, an autonomous agent interacts with an external environment by selecting actions and seeks the sequence of actions that maximizes its long-term performance. In reinforcement learning, the environment is typically initially unknown and learning takes place online, that is, the agent’s performance is assessed throughout learning instead of only afterwards. Since many challenging and realistic tasks are well described as sequential decision problems, the development of effective reinforcement-learning algorithms plays an important role in artificial intelligence research. Recent years have seen enormous progress, both in the development of new methods and the theoretical understanding of existing methods. In particular, great strides have been made in approximating value functions, exploring efficiently, learning in the presence of multiple agents, coping with partial observability, inducing models, and reasoning hierarchically. The focus of this special issue is not the development of new algorithms but the empirical evaluation of existing ones. Like other machine-learning methods, reinforcement-learning approaches are typically evaluated in one or more of the following three ways: (1) subjectively, (2) theoretically, and (3) empirically. Subjective evaluations, in which researchers assess the significance and potential of new ideas, are important because they leverage the powerful intuition of experts to guide the research process. However, they are also limited because they cannot validate ideas that go against such intuition; for example, they cannot expose fallacious assumptions. Theoretical evaluations are also important, as they are
[1]
Adele E. Howe,et al.
How evaluation guides AI research
,
1988
.
[2]
Sebastian Thrun,et al.
Winning the DARPA Grand Challenge with an AI Robot
,
2006,
AAAI.
[3]
Shimon Whiteson,et al.
Generalized Domains for Empirical Evaluations in Reinforcement Learning
,
2009
.
[4]
Shimon Whiteson,et al.
Protecting against evaluation overfitting in empirical reinforcement learning
,
2011,
2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).
[5]
Herbert A. Simon,et al.
Artificial Intelligence: An Empirical Science
,
1995,
Artif. Intell..
[6]
Brian Tanner,et al.
RL-Glue: Language-Independent Software for Reinforcement-Learning Experiments
,
2009,
J. Mach. Learn. Res..
[7]
Pat Langley,et al.
Machine learning as an experimental science
,
2004,
Machine Learning.
[8]
Erik Blasch,et al.
Unmanned vehicles come of age: The DARPA grand challenge
,
2006,
Computer.
[9]
Ronald E. Parr,et al.
A Novel Benchmark Methodology and Data Repository for Real-life Reinforcement Learning
,
2009
.
[10]
Yehuda Koren,et al.
All Together Now: A Perspective on the Netflix Prize
,
2010
.
[11]
Shimon Whiteson,et al.
The Reinforcement Learning Competitions
,
2010
.