Portability of Semantic Annotations for Fast Development of Dialogue Corpora

Generalization of spoken dialogue systems increases the need for fast development of spoken language understanding modules for semantic tagging of speaker’s turns. Statistical methods are performing well for this task but require large corpora to be trained. Collecting such corpora is expensive in time and human expertise. In this paper we propose a semi-automatic annotation process for fast production of dialogue corpora by automatically pre-annotating the corpus before performing manual corrections. For the pre-annotation we propose to port a system initiated from an existing corpus and to adapt it to the new data. The French MEDIA dialogue corpus (hotel reservation) is used as a starting point to produce two new corpora: one for a new language (Italian) and another for a new domain (theatre ticket reservation). We show that the automatic pre-annotation leads to a significant gain in productivity compared to a fully manual annotation and thus allows to derive adaptation data which can be used in turn to further improve the systems.

[1]  Hermann Ney,et al.  Comparing Stochastic Approaches to Spoken Language Understanding in Multiple Languages , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[2]  Ruhi Sarikaya,et al.  Rapid bootstrapping of statistical spoken dialogue systems , 2008, Speech Commun..

[3]  Sophie Rosset,et al.  Semantic annotation of the French media dialog corpus , 2005, INTERSPEECH.

[4]  Ben Taskar,et al.  Alignment by Agreement , 2006, NAACL.

[5]  Philipp Koehn,et al.  Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[6]  David Griol,et al.  A statistical approach to spoken dialog systems design and evaluation , 2008, Speech Commun..

[7]  Fabrice Lef.  DYNAMIC BAYESIAN NETWORKS AND DISCRIMINATIVE CLASSIFIERS FOR MULTI-STAGE SEMANTIC INTERPRETATION , 2007 .

[8]  François Yvon,et al.  Practical Very Large Scale CRFs , 2010, ACL.

[9]  Alex Acero,et al.  Discriminative models for spoken language understanding , 2006, INTERSPEECH.

[10]  Gökhan Tür,et al.  Combining active and semi-supervised learning for spoken language understanding , 2005, Speech Commun..

[11]  Fabrice Lefèvre,et al.  Investigating multiple approaches for SLU portability to a new language , 2010, INTERSPEECH.

[12]  Fabrice Lefèvre,et al.  Cross-lingual spoken language understanding from unaligned data using discriminative classification models and machine translation , 2010, INTERSPEECH.

[13]  Frédéric Béchet,et al.  On the use of machine translation for spoken language understanding portability , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  María Inés Torres,et al.  EDECÁN: sistEma de Diálogo multidominio con adaptación al contExto aCústico y de AplicacióN , 2006 .

[15]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[16]  Benoît Favre,et al.  Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the PORTMEDIA corpora , 2012, LREC.

[17]  Milica Gasic,et al.  Natural belief-critic: a reinforcement algorithm for parameter estimation in statistical spoken dialogue systems , 2010, INTERSPEECH.

[18]  Kallirroi Georgila,et al.  EVALUATING EFFECTIVENESS AND PORTABILITY OF REINFORCEMENT LEARNED DIALOGUE STRATEGIES WITH REAL USERS: THE TALK TOWNINFO EVALUATION , 2006, 2006 IEEE Spoken Language Technology Workshop.

[19]  Liang Gu,et al.  Portability challenges in developing interactive dialogue systems , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[20]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[21]  David Griol,et al.  A stochastic finite-state transducer approach to spoken dialog management , 2010, INTERSPEECH.