Conversational Search with Mixed-Initiative - Asking Good Clarification Questions backed-up by Passage Retrieval

We deal with a scenario of conversational search with mixed-initiative: namely user-asks system-answers, as well as system-asks (clarification questions) and user-answers. We focus on the task of selecting the next clarification question, given conversation context. Our method leverages passage retrieval that is used both for an initial selection of relevant candidate clarification questions, as well as for fine-tuning two deep-learning models for reranking these candidates. We evaluated our method on two different use-cases. The first is an open domain conversational search in a large web collection. The second is a taskoriented customer-support setup. We show that our method performs well on both usecases.

[1]  Paul N. Bennett,et al.  Generating Clarifying Questions for Information Retrieval , 2020, WWW.

[2]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[3]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[4]  David Konopnicki,et al.  Conversational Document Prediction to Assist Customer Care Agents , 2020, EMNLP.

[5]  David Konopnicki,et al.  Queries as anchors: selection by association , 2005, HYPERTEXT '05.

[6]  Evangelos Kanoulas,et al.  Analysing the Effect of Clarifying Questions on Document Ranking in Conversational Search , 2020, ICTIR.

[7]  Hugo Zaragoza,et al.  The Probabilistic Relevance Framework: BM25 and Beyond , 2009, Found. Trends Inf. Retr..

[8]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[9]  David Konopnicki,et al.  A Study of BERT for Non-Factoid Question-Answering under Passage Length Constraints , 2019, ArXiv.

[10]  Mark Hopkins,et al.  Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples , 2018, ACL.

[11]  Mikhail Burtsev,et al.  ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ) , 2020, ArXiv.

[12]  W. Bruce Croft,et al.  Asking Clarifying Questions in Open-Domain Information-Seeking Conversations , 2019, SIGIR.

[13]  Hal Daumé,et al.  Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information , 2018, ACL.

[14]  Douglas W. Oard,et al.  A Fixed-Point Method for Weighting Terms in Verbose Informational Queries , 2014, CIKM.

[15]  Yue Lin,et al.  A Clarifying Question Selection System from NTES_ALONG in Convai3 Challenge , 2020, ArXiv.