Conversational Search - A Report from Dagstuhl Seminar 19461

Dagstuhl Seminar 19461 "Conversational Search" was held on 10-15 November 2019. 44~researchers in Information Retrieval and Web Search, Natural Language Processing, Human Computer Interaction, and Dialogue Systems were invited to share the latest development in the area of Conversational Search and discuss its research agenda and future directions. A 5-day program of the seminar consisted of six introductory and background sessions, three visionary talk sessions, one industry talk session, and seven working groups and reporting sessions. The seminar also had three social events during the program. This report provides the executive summary, overview of invited talks, and findings from the seven working groups which cover the definition, evaluation, modelling, explanation, scenarios, applications, and prototype of Conversational Search. The ideas and findings presented in this report should serve as one of the main sources for diverse research programs on Conversational Search.

[1]  Jimmy J. Lin,et al.  Overview of the TREC 2006 ciQA task , 2007, SIGF.

[2]  Benno Stein,et al.  Argumentation Synthesis following Rhetorical Strategies , 2018, COLING.

[3]  Ben Carterette,et al.  TREC 2017 Tasks Track Overview , 2017, TREC.

[4]  Ragnar Nordlie,et al.  “User revealment”—a comparison of initial queries and ensuing question development in online searching and in human reference interactions , 1999, SIGIR '99.

[5]  Mitesh M. Khapra,et al.  Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph , 2018, AAAI.

[6]  Johanne R. Trippas,et al.  Spoken Conversational Search: Audio-only Interactive Information Retrieval , 2019 .

[7]  Danqi Chen,et al.  CoQA: A Conversational Question Answering Challenge , 2018, TACL.

[8]  Hung-Hsuan Huang,et al.  Embodied Conversational Agents , 2009 .

[9]  Eunsol Choi,et al.  QuAC: Question Answering in Context , 2018, EMNLP.

[10]  Jeff Dalton,et al.  Vote Goat: Conversational Movie Recommendation , 2018, SIGIR.

[11]  Jutta Haider,et al.  Invisible Search and Online Search Engines , 2019 .

[12]  Sören Auer,et al.  Open Research Knowledge Graph: Next Generation Infrastructure for Semantic Scholarly Knowledge , 2019, K-CAP.

[13]  Jens Edlund,et al.  The State of Speech in HCI: Trends, Themes and Challenges , 2018, Interact. Comput..

[14]  Ryen W. White,et al.  Exploratory Search: Beyond the Query-Response Paradigm , 2009, Exploratory Search: Beyond the Query-Response Paradigm.

[15]  M. de Rijke,et al.  QRFA: A Data-Driven Model of Information-Seeking Dialogues , 2018, ECIR.

[16]  M. D. Rijke,et al.  Information Retrieval Evaluation in a Changing World: Lessons Learned from 20 Years of CLEF , 2019, Information Retrieval Evaluation in a Changing World.

[17]  Michael F. McTear,et al.  Book Review: Spoken Dialogue Technology: Toward the Conversational User Interface, by Michael F. McTear , 2002, CL.

[18]  Jimmy J. Lin,et al.  Evaluation-as-a-Service for the Computational Sciences , 2018, ACM J. Data Inf. Qual..

[19]  Ryen W. White,et al.  Supporting Complex Search Tasks , 2014, CIKM.

[20]  Tefko Saracevic,et al.  Challenges for Digital Library Evaluation. , 2000 .

[21]  Matthias Hagen,et al.  Modeling Deliberative Argumentation Strategies on Wikipedia , 2018, ACL.

[22]  Harry Shum,et al.  From Eliza to XiaoIce: challenges and opportunities with social chatbots , 2018, Frontiers of Information Technology & Electronic Engineering.

[23]  W. Bruce Croft,et al.  Neural Matching Models for Question Retrieval and Next Question Prediction in Conversation , 2017, ArXiv.

[24]  Hamed Zamani,et al.  Macaw: An Extensible Conversational Information Seeking Platform , 2019, SIGIR.

[25]  Gerhard Weikum,et al.  Look before you Hop: Conversational Question Answering over Knowledge Graphs Using Judicious Context Expansion , 2019, CIKM.

[26]  Filip Radlinski,et al.  Common Conversational Community Prototype: Scholarly Conversational Assistant , 2020, ArXiv.

[27]  Qifa Ke,et al.  Conversational Query Understanding Using Sequence to Sequence Modeling , 2018, WWW.

[28]  Peter Clark,et al.  Learning Knowledge Graphs for Question Answering through Conversational Dialog , 2015, NAACL.

[29]  Sarah Sharples,et al.  Voice Interfaces in Everyday Life , 2018, CHI.

[30]  Sarah Sharples,et al.  "Do Animals Have Accents?": Talking with Agents in Multi-Party Conversation , 2017, CSCW.

[31]  W. Bruce Croft,et al.  Asking Clarifying Questions in Open-Domain Information-Seeking Conversations , 2019, SIGIR.

[32]  Masatoshi Yoshikawa,et al.  Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training , 2018, ACM Multimedia.

[33]  Martin Halvey,et al.  Conceptualizing agent-human interactions during the conversational search process , 2018 .

[34]  Emine Yilmaz,et al.  Research Frontiers in Information Retrieval Report from the Third Strategic Workshop on Information Retrieval in Lorne (SWIRL 2018) , 2018 .

[35]  Iryna Gurevych,et al.  When is ACL's Deadline? A Scientific Conversational Agent , 2019, ArXiv.

[36]  Martha Larson,et al.  Continuous Evaluation of Large-Scale Information Access Systems: A Case for Living Labs , 2019, Information Retrieval Evaluation in a Changing World.

[37]  Catherine Pelachaud,et al.  From brows to trust: evaluating embodied conversational agents , 2004 .

[38]  Sharon Oviatt,et al.  The Design of Future Educational Interfaces , 2013 .

[39]  Svitlana Vakulenko,et al.  Knowledge-based Conversational Search , 2019, ArXiv.

[40]  Mark G. Core,et al.  Coding Dialogs with the DAMSL Annotation Scheme , 1997 .

[41]  Guillaume Bouchard,et al.  Interpretation of Natural Language Rules in Conversational Machine Reading , 2018, EMNLP.

[42]  W. Bruce Croft,et al.  User Intent Prediction in Information-seeking Conversations , 2019, CHIIR.

[43]  Diego Reforgiato Recupero,et al.  Conference Live: Accessible and Sociable Conference Semantic Data , 2015, WWW.

[44]  Mark Sanderson,et al.  How Do People Interact in Conversational Speech-Only Search Tasks: A Preliminary Analysis , 2017, CHIIR.

[45]  Peter F. Patel-Schneider,et al.  Voice in the User Interface , 2014 .

[46]  James Allan,et al.  Frontiers, challenges, and opportunities for information retrieval: Report from SWIRL 2012 the second strategic workshop on information retrieval in Lorne , 2012, SIGF.

[47]  Jun Huang,et al.  Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems , 2018, SIGIR.

[48]  Susan Brennan,et al.  Interaction and feedback in a spoken language system: a theoretical framework , 1995, Knowl. Based Syst..

[49]  Benno Stein,et al.  TIRA Integrated Research Architecture , 2019, Information Retrieval Evaluation in a Changing World.

[50]  Harry Shum,et al.  The Design and Implementation of XiaoIce, an Empathetic Social Chatbot , 2018, CL.

[51]  Adelheit Stein,et al.  Structuring collaborative information-seeking dialogues , 1995, Knowl. Based Syst..

[52]  Filip Radlinski,et al.  Towards Conversational Recommender Systems , 2016, KDD.

[53]  Kevyn Collins-Thompson,et al.  Search as Learning (Dagstuhl Seminar 17092) , 2017, Dagstuhl Reports.

[54]  Robert N. Oddy,et al.  INFORMATION RETRIEVAL THROUGH MAN‐MACHINE DIALOGUE , 1977 .

[55]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[56]  Adrian Hernandez-Mendez,et al.  Evaluating Natural Language Understanding Services for Conversational Question Answering Systems , 2017, SIGDIAL Conference.

[57]  Ahmed Elgohary,et al.  A dataset and baselines for sequential open-domain question answering , 2018, EMNLP.

[58]  Alan W. Black,et al.  A Dataset for Document Grounded Conversations , 2018, EMNLP.

[59]  A. Avramides.  Studies in the Way of Words , 1992 .

[60]  Jason Weston,et al.  Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks , 2015, ICLR.

[61]  Krisztian Balog,et al.  Personal Knowledge Graphs: A Research Agenda , 2019, ICTIR.

[62]  Idan Szpektor,et al.  Novelty based Ranking of Human Answers for Community Questions , 2016, SIGIR.

[63]  Jonas Beskow,et al.  Developing and evaluating conversational agents , 2001 .

[64]  Ming-Wei Chang,et al.  Search-based Neural Structured Learning for Sequential Question Answering , 2017, ACL.

[65]  Marilyn A. Walker,et al.  PARADISE: A Framework for Evaluating Spoken Dialogue Agents , 1997, ACL.

[66]  Filip Radlinski,et al.  A Theoretical Framework for Conversational Search , 2017, CHIIR.

[67]  Norbert Fuhr,et al.  Some Common Mistakes In IR Evaluation, And How They Can Be Avoided , 2018, SIGIR Forum.

[68]  Gaynor Williams.  Space to Think , 2004 .

[69]  Hideo Joho,et al.  Towards a Model for Spoken Conversational Search , 2019, Inf. Process. Manag..

[70]  J. V. Kuppevelt,et al.  Advances in natural multimodal dialogue systems , 2005 .

[71]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[72]  Sharon L. Oviatt,et al.  The Paradigm Shift to Multimodality in Contemporary Computer Interfaces , 2015, Synthesis Lectures on Human-Centered Informatics.

[73]  Mike Thelwall,et al.  Synthesis Lectures on Information Concepts, Retrieval, and Services , 2009 .

[74]  Xu Chen,et al.  Towards Conversational Search and Recommendation: System Ask, User Respond , 2018, CIKM.

[75]  Antonio Krüger,et al.  The Handbook of Multimodal-Multisensor Interfaces: Foundations, User Modeling, and Common Modality Combinations - Volume 1 , 2017, The Handbook of Multimodal-Multisensor Interfaces, Volume 1.

[76]  Mark Sanderson,et al.  Informing the Design of Spoken Conversational Search: Perspective Paper , 2018, CHIIR.

[77]  Waleed Ammar,et al.  GrapAL: Querying Semantic Scholar's Literature Graph , 2019, ArXiv.

[78]  Grace Hui Yang,et al.  Dynamic Search - Optimizing the Game of Information Seeking , 2019, ArXiv.

[79]  Ludovic Denoyer,et al.  A Reinforcement Learning-driven Translation Model for Search-Oriented Conversational Systems , 2018, SCAI@EMNLP.

[80]  J. Cassell,et al.  SOCIAL DIALOGUE WITH EMBODIED CONVERSATIONAL AGENTS , 2005 .

[81]  Jing Zhang,et al.  AMiner: Search and Mining of Academic Social Networks , 2019, Data Intelligence.