Towards an open-domain conversational system fully based on natural language processing

This paper proposes an architecture for an open-domain conversational system and evaluates an implemented system. The proposed architecture is fully composed of modules based on natural language processing techniques. Experimental results using human subjects show that our architecture achieves significantly better naturalness than a retrieval-based baseline and that its naturalness is close to that of a rule-based system using 149K hand-crafted rules.

[1]  Marti A. Hearst.  Automatic Acquisition of Hyponyms from Large Text Corpora , 1992, COLING.

[2]  Oren Etzioni,et al.  Open Information Extraction from the Web , 2007, CACM.

[3]  E. Schegloff,et al.  Opening up Closings , 1973 .

[4]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[5]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[6]  Reid Swanson,et al.  Say Anything: Using Textual Case-Based Reasoning to Enable Open-Domain Interactive Storytelling , 2012, TIIS.

[7]  Ryuichiro Higashinaka,et al.  Automatically Acquiring Causal Expression Patterns from Relation-annotated Corpora to Improve Question Answering for why-Questions , 2008, TALIP.

[8]  S. Eggins,et al.  Analysing Casual Conversation , 1996 .

[9]  Ryuichiro Higashinaka,et al.  Large-scale Collection and Analysis of Personal Question-Answer Pairs for Conversational Agents , 2014, IVA.

[10]  Kiyohiro Shikano,et al.  Construction and Optimization of a Question and Answer Database for a Real-environment Speech-oriented Guidance System , 2007 .

[11]  Y. Wilks,et al.  CONVERSE: a Conversational Companion , 1999 .

[12]  Yasuo Kuniyoshi,et al.  Dialog System Using Real-Time Crowdsourcing and Twitter Large-Scale Corpus , 2012, SIGDIAL Conference.

[13]  Justine Cassell,et al.  "How about this weather?" Social Dialogue with Embodied Conversational Agents , 2000 .

[14]  Philipp Koehn,et al.  Synthesis Lectures on Human Language Technologies , 2016 .

[15]  Mirella Lapata,et al.  Probabilistic Text Structuring: Experiments with Sentence Ordering , 2003, ACL.

[16]  Richard S. Wallace,et al.  The Anatomy of A.L.I.C.E. , 2009 .

[17]  Ryuichiro Higashinaka,et al.  Learning to control listening-oriented dialogue using partially observable markov decision processes , 2013, TSLP.

[18]  Julia Hirschberg,et al.  High Frequency Word Entrainment in Spoken Dialogue , 2008, ACL.

[19]  Kenji Araki,et al.  A Casual Conversation System Using Modality and Word Associations Retrieved from the Web , 2008, EMNLP.

[20]  Alan Ritter,et al.  Data-Driven Response Generation in Social Media , 2011, EMNLP.

[21]  Ryuichiro Higashinaka,et al.  Open-domain Utterance Generation for Conversational Dialogue Systems using Web-scale Dependency Structures , 2013, SIGDIAL Conference.

[22]  Eduard Hovy,et al.  Approaches to the Planning of Coherent Text , 1991 .

[23]  Mikio Nakano,et al.  WIT: A Toolkit for Building Robust and Real-Time Spoken Dialogu Systems , 2000, SIGDIAL Workshop.

[24]  Ryuichiro Higashinaka,et al.  Syntactic Filtering and Content-Based Retrieval of Twitter Sentences for the Generation of System Utterances in Dialogue Systems , 2016 .

[25]  Vernon Boes,et al.  Intentional Interviewing and Counseling: Facilitating Client Development in a Multicultural Society, Seventh Edition , 2010 .

[26]  Marilyn A. Walker,et al.  Towards developing general models of usability with PARADISE , 2000, Natural Language Engineering.

[27]  Ellen M. Voorhees,et al.  Overview of the TREC-9 Question Answering Track , 2000, TREC.

[28]  Timothy W. Bickmore,et al.  Establishing and maintaining long-term human-computer relationships , 2005, TCHI.

[29]  Marilyn A. Walker,et al.  Japanese Discourse and the Process of Centering , 1994, Comput. Linguistics.

[30]  Antoine Raux,et al.  The Dialog State Tracking Challenge , 2013, SIGDIAL Conference.

[31]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[32]  Marilyn A. Walker,et al.  Quantitative and Qualitative Evaluation of Darpa Communicator Spoken Dialogue Systems , 2001, ACL.

[33]  Masahiro Shibata,et al.  Dialog System for Open-Ended Conversation Using Web Documents , 2009, Informatica.

[34]  Tomoko Izumi,et al.  Discriminative Approach to Predicate-Argument Structure Analysis with Zero-Anaphora Resolution , 2009, ACL.

[35]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[36]  Tomoko Izumi,et al.  Predicate-Argument Structure Analysis with Zero-Anaphora Resolution for Dialogue Systems , 2014, COLING.

[37]  Cícero Nogueira dos Santos,et al.  Semantic Role Labeling , 2012 .