Big Data for Conversational Interfaces: Current Opportunities and Prospects

As conversational technologies develop, we demand more from them. For instance, we want our conversational assistants to be able to solve our queries in multiple domains, to manage information from different usually unstructured sources, to be able to perform a variety of tasks, and understand open conversational language. However, developing the resources necessary to develop systems with such capabilities demands much time and effort, as for each domain, task or language, data must be collected, annotated following an schema that is usually not portable, the models must be trained over the annotated data, and their accuracy must be evaluated. In recent years, there has been a growing interest in investigating alternatives to manual effort that allow exploiting automatically the huge amount of resources available in the web. In this chapter we describe the main initiatives to extract, process and contextualize information from these rich and heterogeneous sources for the various tasks involved in dialog systems, including speech processing, natural language understanding and dialog management.

[1]  Douglas D. O'Shaughnessy,et al.  Invited paper: Automatic speech recognition: History, methods and challenges , 2008, Pattern Recognit..

[2]  Feng Gao,et al.  Spoken language understanding using weakly supervised learning , 2010, Comput. Speech Lang..

[3]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[4]  Slim Abdennadher,et al.  BECAM tool - a semi-automatic tool for bootstrapping emotion corpus annotation and management , 2007, INTERSPEECH.

[5]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[6]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[7]  Nigel Gilbert,et al.  Simulating speech systems , 1991 .

[8]  Frédéric Béchet,et al.  Conceptual decoding for spoken dialog systems , 2003, INTERSPEECH.

[9]  Christopher Rose,et al.  Editorial: Mobility Management , 1996, Mob. Networks Appl..

[10]  Andreas Stolcke,et al.  Does active learning help automatic dialog act tagging in meeting data? , 2005, INTERSPEECH.

[11]  Dafydd Gibbon,et al.  Handbook of Multimodal and Spoken Dialogue Systems , 2000 .

[12]  Satoshi Nakamura,et al.  Recent advances in WFST-based dialog system , 2009, INTERSPEECH.

[13]  Alexander I. Rudnicky,et al.  Stochastic Language Generation for Spoken Dialogue Systems , 2000 .

[14]  S. Singh,et al.  Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System , 2011, J. Artif. Intell. Res..

[15]  Lin-Shan Lee,et al.  Computer-aided analysis and design for spoken dialogue systems based on quantitative simulations , 2001, IEEE Trans. Speech Audio Process..

[16]  Taghi M. Khoshgoftaar,et al.  Deep learning applications and challenges in big data analytics , 2015, Journal of Big Data.

[17]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[18]  Matthias Steinbauer,et al.  Using Big Data for Emotionally Intelligent Mobile Services through Multi-Modal Emotion Recognition , 2015, ICOST.

[19]  Encarna Segarra,et al.  User simulation in a stochastic dialog system , 2008, Comput. Speech Lang..

[20]  Roger K. Moore,et al.  Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation , 2000 .

[21]  Andreas Stolcke,et al.  AUTOMATIC DIALOG ACT LABELING WITH MINIMAL SUPERVISION , 2008 .

[22]  Wolfgang Minker,et al.  Design considerations for knowledge source representations of a stochastically-based natural language understanding component , 1999, Speech Commun..

[23]  Wolfgang Minker,et al.  Introducing Spoken Dialogue Systems into Intelligent Environments , 2012 .

[24]  Steve Young,et al.  The statistical approach to the design of spoken dialogue systems , 2003 .

[25]  Joelle Pineau,et al.  Spoken Dialogue Management Using Probabilistic Reasoning , 2000, ACL.

[26]  Yorick Wilks,et al.  Some background on dialogue management and conversational speech for dialogue systems , 2011, Comput. Speech Lang..

[27]  Grace Chung,et al.  Developing a Flexible Spoken Dialog System Using Simulation , 2004, ACL.

[28]  Roberto Pieraccini,et al.  User modeling for spoken dialogue system evaluation , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[29]  Encarna Segarra,et al.  Development of a stochastic dialog manager driven by semantics , 2003, INTERSPEECH.

[30]  Toni Giorgino,et al.  Adaptable dialog architecture and runtime engine (AdaRTE): A framework for rapid prototyping of health dialog systems , 2009, Int. J. Medical Informatics.

[31]  John Mourjopoulos,et al.  Automatic speech recognition performance in different room acoustic environments with and without dereverberation preprocessing , 2013, Comput. Speech Lang..

[32]  Thomas Hempel.  Usability of Speech Dialog Systems: Listening to the Target Audience , 2010 .

[33]  Ramón López-Cózar,et al.  ASR post-correction for spoken dialogue systems based on semantic, syntactic, lexical and contextual information , 2008, Speech Commun..

[34]  Kevin Hannam,et al.  Applied mobilities, transitions and opportunities , 2016 .

[35]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[36]  Hui Ye,et al.  Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System , 2007, NAACL.

[37]  Alex Waibel,et al.  Stochastically-Based Semantic Analysis , 1999 .

[38]  S. Renals,et al.  Spoken dialogue interfaces for older people , 2012 .

[39]  Roberto Pieraccini,et al.  Automating spoken dialogue management design using machine learning: An industry perspective , 2008, Speech Commun..

[40]  Roberto Pieraccini.  The Voice in the Machine: Building Computers That Understand Speech , 2012 .

[41]  Encarna Segarra,et al.  An Online Generated Transducer to Increase Dialog Manager Coverage , 2012, INTERSPEECH.

[42]  Pascal Poupart,et al.  Partially Observable Markov Decision Processes with Continuous Observations for Dialogue Management , 2008, SIGDIAL.

[43]  David Traum,et al.  The Information State Approach to Dialogue Management , 2003 .

[44]  Biing-Hwang Juang,et al.  An Overview of Automatic Speech Recognition , 1996 .

[45]  Hui Ye,et al.  Training a real-world POMDP-based Dialog System , 2007, HLT-NAACL 2007.

[46]  Dong Yu,et al.  Conversational Speech Transcription Using Context-Dependent Deep Neural Networks , 2012, ICML.

[47]  Sebastian Möller,et al.  Memo: towards automatic usability evaluation of spoken dialogue services by user error simulations , 2006, INTERSPEECH.

[48]  Frederick Jelinek,et al.  Basic Methods of Probabilistic Context Free Grammars , 1992 .

[49]  David Suendermann-Oeft,et al.  One Year of Contender: What Have We Learned about Assessing and Tuning Industrial Spoken Dialog Systems? , 2012, SDCTD@NAACL-HLT.

[50]  David Suendermann,et al.  Data-Driven Methods in Industrial Spoken Dialog Systems , 2012 .

[51]  Ramón López-Cózar,et al.  Assessment of dialogue systems by means of a new simulation technique , 2003, Speech Commun..

[52]  Steve Young,et al.  Statistical User Simulation with a Hidden Agenda , 2007, SIGDIAL.

[53]  Ramón López-Cózar,et al.  Testing the performance of spoken dialogue systems by means of an artificially simulated user , 2006, Artificial Intelligence Review.

[54]  Lalit R. Bahl,et al.  A Maximum Likelihood Approach to Continuous Speech Recognition , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Oliver Lemon,et al.  Requirements analysis and theory for statistical learning approaches in automaton-based dialogue management , 2014 .

[56]  Marilyn A. Walker,et al.  Reinforcement Learning for Spoken Dialogue Systems , 1999, NIPS.

[57]  Elmar Nöth,et al.  A Taxonomy of Applications that Utilize Emotional Awareness , 2006 .

[58]  Diane J. Litman,et al.  Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors , 2006, Speech Commun..

[59]  David Griol,et al.  The Conversational Interface: Talking to Smart Devices , 2016 .

[60]  Peter A. Heeman.  Combining Reinformation Learning with Information-State Update Rules , 2007, HLT-NAACL.

[61]  Kallirroi Georgila,et al.  Learning user simulations for information state update dialogue systems , 2005, INTERSPEECH.

[62]  Viktor Mayer-Schnberger,et al.  Big Data: A Revolution That Will Transform How We Live, Work, and Think , 2013 .

[63]  Frederick Jelinek,et al.  Self-organizing language modeling for speech recognition , 1990 .

[64]  Chunhua Weng,et al.  Leveraging dialog systems research to assist biomedical researchers' interrogation of Big Clinical Data , 2016, J. Biomed. Informatics.

[65]  Roberto Pieraccini,et al.  Concept-based spontaneous speech understanding system , 1995, EUROSPEECH.

[66]  Oliver Lemon,et al.  Learning what to say and how to say it: Joint optimisation of spoken dialogue management and natural language generation , 2011, Comput. Speech Lang..

[67]  Gökhan Tür,et al.  Bootstrapping spoken dialogue systems by exploiting reusable libraries , 2008, Nat. Lang. Eng..

[68]  Eric Horvitz,et al.  Conversation as Action Under Uncertainty , 2000, UAI.

[69]  Vijay V. Raghavan,et al.  Big Data Driven Natural Language Processing Research and Applications , 2015 .

[70]  Timothy Bickmore,et al.  Some Novel Aspects of Health Communication from a Dialogue Systems Perspective , 2004, AAAI Technical Report.

[71]  Zuhair Bandar,et al.  A Multi-classifier Approach to Dialogue Act Classification Using Function Words , 2012, Trans. Comput. Collect. Intell..

[72]  Encarna Segarra,et al.  Extracting Semantic Information Through Automatic Learning Techniques , 2002, Int. J. Pattern Recognit. Artif. Intell..

[73]  Gary Geunbae Lee,et al.  Hybrid approach to robust dialog management using agenda and dialog examples , 2010, Comput. Speech Lang..

[74]  Juan Luis Castro,et al.  A case based reasoning model for multilingual language generation in dialogues , 2012, Expert Syst. Appl..

[75]  Gary Geunbae Lee,et al.  Hybrid user intention modeling to diversify dialog simulations , 2011, Comput. Speech Lang..

[76]  H. Cuayahuitl,et al.  Human-computer dialogue simulation using hidden Markov models , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[77]  David Griol,et al.  A stochastic finite-state transducer approach to spoken dialog management , 2010, INTERSPEECH.

[78]  Xabier Artola,et al.  Big data for Natural Language Processing: A streaming approach , 2015, Knowl. Based Syst..

[79]  Steve Young,et al.  A data-driven spoken language understanding system , 2003, 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721).

[80]  Kallirroi Georgila,et al.  EVALUATING EFFECTIVENESS AND PORTABILITY OF REINFORCEMENT LEARNED DIALOGUE STRATEGIES WITH REAL USERS: THE TALK TOWNINFO EVALUATION , 2006, 2006 IEEE Spoken Language Technology Workshop.

[81]  Oliver Lemon,et al.  DIPPER: Description and Formalisation of an Information-State Update Dialogue System Architecture , 2003, SIGDIAL Workshop.

[82]  Hui Ye,et al.  The Hidden Information State Approach to Dialog Management , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[83]  Roberto Pieraccini,et al.  The use of belief networks for mixed-initiative dialog modeling , 2000, IEEE Trans. Speech Audio Process..

[84]  M. Shamim Hossain,et al.  Audio-Visual Emotion Recognition Using Big Data Towards 5G , 2016, Mob. Networks Appl..

[85]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[86]  Oliver Lemon,et al.  Data-Driven Methods for Adaptive Spoken Dialogue Systems , 2012, Springer New York.

[87]  Ramón López-Cózar,et al.  Using knowledge of misunderstandings to increase the robustness of spoken dialogue systems , 2010, Knowl. Based Syst..

[88]  Jason D. Williams,et al.  The best of both worlds: unifying conventional dialog systems and POMDPs , 2008, INTERSPEECH.