Evaluation of the Turn Planner in CIRCSIM-Tutor

CIRCSIM-Tutor is an intelligent tutoring system designed to imitate human tutorial dialogue. The turn planner is a planning module intended to be plugged into the forthcoming new version that improves dialogue coherence by post-processing the collection of sentences that constitute a single dial ogue turn. In order to know how much the turn planner improves the dialogue, we are now facing the issue of how to evaluate the contribution of turn planning to the whole system. In this paper we propose a method to evaluate the adequacy from the user’s perspective of the turn planner’s changes to the text.