A Reinforcement Learning approach to evaluating state representations in spoken dialogue systems

Although dialogue systems have been an area of research for decades, finding accurate ways of evaluating different systems is still a very active subfield since many leading methods, such as task completion rate or user satisfaction, capture different aspects of the end-to-end human-computer dialogue interaction. In this work, we step back the focus from the complete evaluation of a dialogue system to presenting metrics for evaluating one internal component of a dialogue system: its dialogue manager. Specifically, we investigate how to create and evaluate the best state space representations for a Reinforcement Learning model to learn an optimal dialogue control strategy. We present three metrics for evaluating the impact of different state models and demonstrate their use on the domain of a spoken dialogue tutoring system by comparing the relative utility of adding three features to a model of user, or student, state. The motivation for this work is that if one knows which features are best to use, one can construct a better dialogue manager, and thus better performing dialogue systems.