论文信息 - Comparing Action-Query Strategies in Semi-Autonomous Agents

Comparing Action-Query Strategies in Semi-Autonomous Agents

We consider semi-autonomous agents that have uncertain knowledge about their environment, but can ask what action the operator would prefer taking in the current or in a potential future state. Asking queries can help improve behavior, but if queries come at a cost (e.g., due to limited operator attention), the number of queries needs to be minimized. We develop a new algorithm for selecting action queries by adapting the recently proposed Expected Myopic Gain (EMG) from its prior use in settings with reward or transition probability queries to our setting of action queries, and empirically compare it to the current state of the art.

[1] Richard S. Sutton,et al. Introduction to Reinforcement Learning , 1998 .

[2] Daphne Koller,et al. Making Rational Decisions Using Adaptive Utility Elicitation , 2000, AAAI/IAAI.

[3] Michael O. Duff,et al. Design for an Optimal Probe , 2003, ICML.

[4] David A. Cohn,et al. Improving generalization with active learning , 1994, Machine Learning.

[5] Pieter Abbeel,et al. Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[6] Richard S. Sutton,et al. Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7] Eyal Amir,et al. Bayesian Inverse Reinforcement Learning , 2007, IJCAI.

[8] Manuel Lopes,et al. Active Learning for Reward Estimation in Inverse Reinforcement Learning , 2009, ECML/PKDD.

[9] Manuela M. Veloso,et al. Interactive Policy Learning through Confidence-Based Autonomy , 2014, J. Artif. Intell. Res..

[10] Edmund H. Durfee,et al. Selecting Operator Queries Using Expected Myopic Gain , 2010, 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[11] Craig Boutilier,et al. Optimal Bayesian Recommendation Sets and Myopically Optimal Choice Query Sets , 2010, NIPS.