Discuss: toward a domain independent representation of dialogue

While many studies have demonstrated that conversational tutoring systems have a positive effect on learning, the amount of manual effort required to author, design, and tune dialogue behaviors remains a major barrier to widespread deployment and adoption of these systems. Such dialogue systems must not only understand student speech, but must also endeavor to keep students engaged while scaffolding them through the curriculum. Crafting robust, natural tutoring interactions typically involves writing tightly scripted behaviors for a wide variety of student responses and scenarios. Combining statistical machine learning with corpus-based methods in natural language processing presents a possible path to reducing this effort. Advances in reinforcement learning have been applied toward dialogue systems to learn optimal behaviors for a given task. However, these learned dialogue policies are tightly coupled to the specific dialogue system implementation. For content-rich applications such as intelligent tutoring systems, there is an immediate need to learn tutoring strategies and dialogue behaviors that can be leveraged across a variety of materials, concepts and lessons. Further generalization will require an intermediate representation of dialogue that can abstract the conversation to its underlying action, function, and content. This work introduces the Dialogue Schema Unifying Speech and Semantics (DISCUSS), an intermediate linguistic representation that captures the semantics and pragmatics of speech while also allowing for domain-independent modeling of tutorial dialogue. To better understand the benefits of the DISCUSS representation, a corpus of computer-mediated tutorial dialogues was manually tagged with DISCUSS labels. These data were then used for three different tasks: utterance classification, dialogue move selection, and learning gains prediction. System performance in these tasks demonstrate the utility and viability of the DISCUSS representation for analyzing and automating dialogue interactions. Utterance classifiers achieve DISCUSS labeling performance on par with inter-annotator agreement levels. System performance in ranking and selecting follow-up questions illustrates the usefulness of DISCUSS-based features for modeling and identifying the factors behind human decision making when teaching. Correlating features of the dialogue with measured learning gains in students shows how DISCUSS-derived metrics provide a detailed account of real tutoring strategies and student behaviors. Together these results represent a step toward more domain-independent mechanisms for modeling dialogue.