User Modeling Optimization for the Conversational Human-Machine Interfaces

In this paper, we test the applicability of a statistical user modeling technique to develop a simulated user agent that generates dialogs which are similar to real human-machine spoken interactions. The proposed user simulation technique decides the next response of the agent taking into account the previous user turns, the last system answer and the objective of the dialog. In our contribution, we present the results of the comparison between a corpus acquired from real interactions of users with a conversational agent and a corpus acquired by means of the proposed user simulation technique. To do so, we describe the practical application of our proposal for a conversational agent providing tourist information in natural language, employing a comprehensive set of measures for its evaluation.

[1]  David Griol,et al.  Bringing together commercial and academic perspectives for the development of intelligent AmI interfaces , 2012, J. Ambient Intell. Smart Environ..

[2]  William Rand,et al.  An Introduction to Agent-Based Modeling: Modeling Natural, Social, and Engineered Complex Systems with NetLogo , 2015 .

[3]  Roberto Pieraccini,et al.  Automating spoken dialogue management design using machine learning: An industry perspective , 2008, Speech Commun..

[4]  Franziska Klügl-Frohnmeyer,et al.  From Simulated to Real Environments: How to Use SeSAm for Software Development , 2003, MATES.

[5]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[6]  Kallirroi Georgila,et al.  Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems , 2005, SIGDIAL.

[7]  Michael J. North,et al.  Tutorial on agent-based modelling and simulation , 2005, Proceedings of the Winter Simulation Conference, 2005..

[8]  Michael J. North,et al.  Experiences creating three implementations of the repast agent modeling toolkit , 2006, TOMC.

[9]  Jason D. Williams,et al.  The best of both worlds: unifying conventional dialog systems and POMDPs , 2008, INTERSPEECH.

[10]  Raymond R. Hill,et al.  A Survey of Agent-Based Modeling Practices (January 1998 to July 2008) , 2009, J. Artif. Soc. Soc. Simul..

[11]  Hui Ye,et al.  Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System , 2007, NAACL.

[12]  Ramón López-Cózar,et al.  A domain-independent statistical methodology for dialog management in spoken dialog systems , 2014, Comput. Speech Lang..

[13]  Hua Ai,et al.  Comparing Spoken Dialog Corpora Collected with Recruited Subjects versus Real Users , 2007, SIGDIAL.

[14]  Jennifer Balogh,et al.  Voice User Interface Design , 2004 .

[15]  Sebastian Möller,et al.  Memo: towards automatic usability evaluation of spoken dialogue services by user error simulations , 2006, INTERSPEECH.