Large-Scale Acquisition of Commonsense Knowledge via a Quiz Game on a Dialogue System

Commonsense knowledge is essential for fully understanding language in many situations. We acquire large-scale commonsense knowledge from humans using a game with a purpose (GWAP) developed on a smartphone spoken dialogue system. We transform the manual knowledge acquisition process into an enjoyable quiz game and have collected over 150,000 unique commonsense facts by gathering the data of more than 70,000 players over eight months. In this paper, we present a simple method for maintaining the quality of acquired knowledge and an empirical analysis of the knowledge acquisition process. To the best of our knowledge, this is the first work to collect large-scale knowledge via a GWAP on a widely-used spoken dialogue system.

[1]  H. Lieberman.  Common Consensus : a web-based game for collecting commonsense goals , 2007 .

[2]  Gerhard Weikum,et al.  WebChild: harvesting and organizing commonsense knowledge from the web , 2014, WSDM.

[3]  Marco Baroni,et al.  Bootstrapping a Game with a Purpose for Commonsense Collection , 2012, TIST.

[4]  Imed Zitouni,et al.  Automatic Online Evaluation of Intelligent Assistants , 2015, WWW.

[5]  Benjamin Van Durme,et al.  Reporting bias and knowledge acquisition , 2013, AKBC '13.

[6]  Jane Yung-jen Hsu,et al.  Resource-Bounded Crowd-Sourcing of Commonsense Knowledge , 2011, IJCAI.

[7]  Daisuke Kawahara,et al.  Design of Word Association Games using Dialog Systems for Acquisition of Word Association Knowledge , 2016, AKBC@NAACL-HLT.

[8]  Xiang Li,et al.  Commonsense Knowledge Base Completion , 2016, ACL.

[9]  Hugo Liu,et al.  ConceptNet — A Practical Commonsense Reasoning Tool-Kit , 2004 .

[10]  Cungen Cao,et al.  A Survey of Commonsense Knowledge Acquisition , 2013, Journal of Computer Science and Technology.

[11]  吴昱明,et al.  A Survey of Commonsense Knowledge Acquisition , 2013 .

[12]  Manuel Blum,et al.  Verbosity: a game for collecting common-sense facts , 2006, CHI.

[13]  Catherine Havasi,et al.  Representing General Relational Knowledge in ConceptNet 5 , 2012, LREC.

[14]  Roberto Navigli,et al.  Validating and Extending Semantic Knowledge Bases using Video Games with a Purpose , 2014, ACL.

[15]  Qiang Liu,et al.  Aggregating Ordinal Labels from Crowds by Minimax Conditional Entropy , 2014, ICML.

[16]  Jane Yung-jen Hsu,et al.  Community-based game design: experiments on social games for commonsense data collection , 2009, HCOMP '09.

[17]  Douglas B. Lenat,et al.  CYC: a large-scale investment in knowledge infrastructure , 1995, CACM.

[18]  Hayato Kobayashi,et al.  Effects of Game on User Engagement with Spoken Dialogue System , 2015, SIGDIAL Conference.

[19]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[20]  Daisuke Kawahara,et al.  Morphological Analysis for Unsegmented Languages using Recurrent Neural Network Language Model , 2015, EMNLP.

[21]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[22]  Nobuhiro Kaji,et al.  Prediction of Prospective User Engagement with Intelligent Assistants , 2016, ACL.