Spoken Conversational Search: Speech-only Interactive Information Retrieval

This research investigates a new interface paradigm for interactive information retrieval (IIR) which forces us to shift away from the classic "ten blue links" search engine results page. Instead we investigate how to present search results through a conversation over a speech-only communication channel where no screen is available. Accessing information via speech is becoming increasingly pervasive and is already important for people with a visual impairment. However, presenting search results over a speech-only communication channel is challenging due to cognitive limitations and the transient nature of audio. Studies have indicated that the implementation of speech recognizers and screen readers must be carefully designed and cannot simply be added to an existing system. Therefore the aim of this research is to develop a new interaction framework for effective and efficient IIR over a speech-only channel: a Spoken Conversational Search System (SCSS) which provides a conversational approach to defining user information needs, presenting results and enabling search reformulations. In order to contribute to a more efficient and effective search experience when using a SCSS, we intend for a tighter integration between document search and conversational processes.

[1]  Niels Ole Bernsen,et al.  Designing Interactive Speech Systems , 1998, Springer London.

[2]  Jan Noyes,et al.  Workload and the use of automatic speech recognition: The effects of time and resource demands , 1996, Speech Commun..

[3]  Niels Ole Bernsen,et al.  Evaluation and usability of multimodal spoken language dialogue systems , 2004, Speech Commun..

[4]  Nicole Yankelovich,et al.  Designing speech user interfaces , 1998, CHI Conference Summary.

[5]  Markku Turunen,et al.  Evaluation of Mobile and Pervasive Speech Applications , 2012 .

[6]  Yu Shi,et al.  A system for spoken query information retrieval on mobile devices , 2002, IEEE Trans. Speech Audio Process..

[7]  G D Brown,et al.  Word-frequency effects on short-term memory tasks: evidence for a redintegration process in immediate serial recall. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Mark Sanderson,et al.  Results Presentation Methods for a Spoken Conversational Search System , 2015, NWSearch@CIKM.

[9]  Gary Marchionini,et al.  Find What You Need, Understand What You Find , 2007, Int. J. Hum. Comput. Interact..

[10]  Sebastian Varges,et al.  Interactive Question Answering and Constraint Relaxation in Spoken Dialogue Systems , 2006, Natural Language Engineering.

[11]  Amanda Spink,et al.  How are we searching the World Wide Web? A comparison of nine search engine transaction logs , 2006, Inf. Process. Manag..

[12]  Johanna D. Moore,et al.  A Strategy for Information Presentation in Spoken Dialog Systems , 2011, CL.

[13]  James F. Allen,et al.  Toward Conversational Human-Computer Interaction , 2001, AI Mag..

[14]  Vera Demberg,et al.  Linguistic cognitive load : implications for automotive UIs , 2011 .

[15]  Shiri Azenkot,et al.  Exploring the use of speech input by blind people on mobile devices , 2013, ASSETS.

[16]  Mark Sanderson,et al.  Towards Understanding the Impact of Length in Web Search Result Summaries over a Speech-only Communication Channel , 2015, SIGIR.

[17]  Nigel Gilbert,et al.  Simulating speech systems , 1991 .

[18]  J. Lai,et al.  Speech Interface Design , 2006 .

[19]  Diane Kelly,et al.  Methods for Evaluating Interactive Information Retrieval Systems with Users , 2009, Found. Trends Inf. Retr..

[20]  Niels Ole Bernsen,et al.  Designing interactive speech systems - from first ideas to user testing , 1998 .

[21]  Gina-Anne Levow,et al.  Designing SpeechActs: issues in speech user interfaces , 1995, CHI '95.

[22]  Francoise Beaufays,et al.  “Your Word is my Command”: Google Search by Voice: A Case Study , 2010 .

[23]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .