Evaluation of a Fully Automatic Cooperative Persuasive Dialogue System

In this paper, we construct and evaluate a fully automated text-based cooperative persuasive dialogue system, which is able to persuade the user to take a specific action while maintaining user satisfaction. In our previous works, we created a dialogue management module for cooperative persuasive dialogue (Hiraoka et al., Reinforcement learning of cooperative persuasive dialogue policies using framing, Proceedings of international conference on computational linguistics (COLING), 2014), but only evaluated it in a wizard-of-Oz setting, as we did not have the capacity for natural language generation (NLG) and natural language understanding (NLU). In this work, the main technical contribution is the design of the NLU and the NLG modules which allows us to remove this bottleneck and create the first fully automatic cooperative persuasive dialogue system. Based on this system, we performed an evaluation with real users. Experimental results indicate that the learned policy is able to effectively persuade the users: the reward of the proposed model is much higher than baselines and almost the same as a dialogue manager controlled by a human. This tendency is almost the same as our previous evaluation using a wizard-of-Oz framework (Hiraoka et al., Reinforcement learning of cooperative persuasive dialogue policies using framing, Proceedings of international conference on computational linguistics (COLING), 2014), demonstrates that the proposed NLU and NLG modules are effective for cooperative persuasive dialogue.