Measuring Semantic Coherence of a Conversation

Conversational systems have become increasingly popular as a way for humans to interact with computers. To be able to provide intelligent responses, conversational systems must correctly model the structure and semantics of a conversation. We introduce the task of measuring semantic (in)coherence in a conversation with respect to background knowledge, which relies on the identification of semantic relations between concepts introduced during a conversation. We propose and evaluate graph-based and machine learning-based approaches for measuring semantic coherence using knowledge graphs, their vector space embeddings and word embedding models, as sources of background knowledge. We demonstrate how these approaches are able to uncover different coherence patterns in conversations on the Ubuntu Dialogue Corpus.

[1]  Graeme Hirst,et al.  Distributional Measures as Proxies for Semantic Relatedness , 2012, ArXiv.

[2]  Zhongfei Zhang,et al.  Text Coherence Analysis Based on Deep Neural Network , 2017, CIKM.

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  Viktor de Boer,et al.  The knowledge graph as the default data model for learning on heterogeneous knowledge , 2017, Data Sci..

[5]  André Freitas,et al.  Recognizing and Justifying Text Entailment Through Distributional Navigation on Definition Graphs , 2018, AAAI.

[6]  Giuseppe Ottaviano,et al.  Fast and Space-Efficient Entity Linking for Queries , 2015, WSDM.

[7]  Pablo N. Mendes,et al.  Improving efficiency and accuracy in multilingual entity extraction , 2013, I-SEMANTICS '13.

[8]  Joelle Pineau,et al.  Training End-to-End Dialogue Systems with the Ubuntu Dialogue Corpus , 2017, Dialogue Discourse.

[9]  Axel Polleres,et al.  Binary RDF representation for publication and exchange (HDT) , 2013, J. Web Semant..

[10]  Jürgen Umbrich,et al.  Counting to k or how SPARQL1.1 Property Paths Can Be Extended to Top-k Path Queries , 2017, SEMANTICS.

[11]  Heiko Paulheim,et al.  Biased graph walks for RDF graph embeddings , 2017, WIMS.

[12]  Krisztian Balog,et al.  Nordlys: A Toolkit for Entity-Oriented and Semantic Search , 2017, SIGIR.

[13]  Mirella Lapata,et al.  Probabilistic Text Structuring: Experiments with Sentence Ordering , 2003, ACL.

[14]  Axel-Cyrille Ngonga Ngomo,et al.  Enhancing Community Interactions with Data-Driven Chatbots--The DBpedia Chatbot , 2018, WWW.

[15]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[16]  Eric Fosler-Lussier,et al.  Discourse Segmentation of Multi-Party Conversation , 2003, ACL.

[17]  Jens Lehmann,et al.  Neural Network-based Question Answering over Knowledge Graphs on Word and Character Level , 2017, WWW.

[18]  Shafiq R. Joty,et al.  A Neural Local Coherence Model , 2017, ACL.

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[21]  Achim Rettinger,et al.  Towards Holistic Concept Representations: Embedding Relational Knowledge, Visual Attributes, and Distributional Word Semantics , 2017, International Semantic Web Conference.

[22]  Heiko Paulheim,et al.  Global RDF Vector Space Embeddings , 2017, SEMWEB.

[23]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[24]  Joelle Pineau,et al.  The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems , 2015, SIGDIAL Conference.

[25]  Micha Elsner,et al.  Extending the Entity Grid with Entity-Specific Features , 2011, ACL.

[26]  Axel-Cyrille Ngonga Ngomo,et al.  7th Open Challenge on Question Answering over Linked Data (QALD-7) , 2017, SemWebEval@ESWC.