Comparing Reinforcement and Supervised Learning of Dialogue Policies with Real Users

In Chapter 7 we showed that Reinforcement Learning (RL) based strategies can significantly outperform supervised strategies, in interaction with a simulated environment. The ultimate test for dialogue strategies, however, is how they perform with real users. For real users it is often difficult to complete even relatively simple tasks using automated dialogue systems.