Conversational Assistants for Elderly Users - The Importance of Socially Cooperative Dialogue

Conversational agents can provide valuable cognitive and/or emotional assistance to elderly users or people with cognitive impairments who often have difficulties in organizing and following a structured day schedule. Previous research showed that a virtual assistant that can interact in spoken language would be a desirable help for those users. However, these user groups pose specific requirements for spoken dialogue interaction that existing systems hardly meet. This paper presents work on a virtual conversational assistant that was designed for, and together with, elderly as well as cognitively handicapped users. It has been specifically developed to enable ‘socially cooperative dialogue’ – adaptive and aware conversational interaction in which mutual understanding is co-constructed and ensured collaboratively. The technical approach is described and results of evaluation studies are reported.

[1]  Gabriel Skantze.  Error Handling in Spoken Dialogue Systems : Managing Uncertainty, Grounding and Miscommunication , 2007 .

[2]  Stefan Kopp,et al.  Enabling robust and fluid spoken dialogue with cognitively impaired users , 2017, SIGDIAL Conference.

[3]  Juliana Miehle,et al.  A Social Companion and Conversation Partner for Elderly , 2017 .

[4]  Stefan Kopp,et al.  Virtual Agents as Daily Assistants for Elderly or Cognitively Impaired People - Studies on Acceptance and Interaction Feasibility , 2013, IVA.

[5]  Stefan Kopp,et al.  Communicative Listener Feedback in Human-Agent Interaction: Artificial Speakers Need to Be Attentive and Adaptive , 2018, AAMAS.

[6]  Eric Horvitz,et al.  Conversation as Action Under Uncertainty , 2000, UAI.

[7]  Stefan Kopp,et al.  Socially Cooperative Behavior for Artificial Companions for Elderly and Cognitively Impaired People , 2015, ISCT.

[8]  Georgios Meditskos,et al.  KRISTINA: A Knowledge-Based Virtual Conversation Agent , 2017, PAAMS.

[9]  Patrick Thiam,et al.  The SenseEmotion Database: A Multimodal Database for the Development and Systematic Validation of an Automatic Pain- and Emotion-Recognition System , 2016, MPRSS.

[10]  Stefan Kopp,et al.  A conversational virtual human as autonomous assistant for elderly and cognitively impaired users? Social acceptability and design considerations , 2013, GI-Jahrestagung.

[11]  Nicole C. Krämer,et al.  The Effect of an Intelligent Virtual Agent's Nonverbal Behavior with Regard to Dominance and Cooperativity , 2016, IVA.

[12]  Stefan Kopp,et al.  The Communicative Activity of "Making Suggestions" as an Interactional Process: Towards a Dialog Model for HAI , 2017, HAI.

[13]  David R Traum,et al.  Towards a Computational Theory of Grounding in Natural Language Conversation , 1991 .

[14]  Franz Kummert,et al.  Confirmation detection in human-agent interaction using non-lexical speech cues , 2017, ArXiv.

[15]  Karola Pitsch,et al.  Processes of Reminding and Requesting in Supporting People with Special Needs: Human Practices as Basis for Modeling a Virtual Assistant? , 2016, EDIA@ECAI.

[16]  Stefan Kopp,et al.  Middleware for Incremental Processing in Conversational Agents , 2010, SIGDIAL Conference.

[17]  Stefan Kopp,et al.  Pragmatic Multimodality: Effects of Nonverbal Cues of Focus and Certainty in a Virtual Human , 2017, IVA.

[18]  Rieks op den Akker,et al.  A model for incremental grounding in spoken dialogue systems , 2013, Journal on Multimodal User Interfaces.

[19]  Stefan Kopp,et al.  Towards graceful turn management in human-agent interaction for people with cognitive impairments , 2016 .

[20]  Karola Pitsch,et al.  Reprompts as error handling strategy in human-agent-dialog? User responses to a system's display of non-understanding , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[21]  Stefan Kopp,et al.  Analysing the Modifying Functions of Gesture in Multimodal Utterances , 2015 .

[22]  Karola Pitsch,et al.  Dealing with ‘long turns’ produced by users of an assistive system: How missing uptake and recipiency lead to turn increments , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[23]  Nicole C. Krämer,et al.  A Categorization of Virtual Agent Appearances and a Qualitative Study on Age-Related User Preferences , 2017, IVA.

[24]  Karola Pitsch,et al.  Dealing with Long Utterances: How to Interrupt the User in a Socially Acceptable Manner? , 2017, HAI.

[25]  Gabriel Skantze,et al.  A General, Abstract Model of Incremental Dialogue Processing , 2011 .

[26]  Stefan Kopp,et al.  This Is What's Important - Using Speech and Gesture to Create Focus in Multimodal Utterance , 2016, IVA.

[27]  Lin Shi,et al.  Social support agents for older adults: longitudinal affective computing in the home , 2015, Journal on Multimodal User Interfaces.

[28]  Karola Pitsch,et al.  Fallstudien zur Alltagsrelevanz von Zeit- und Kalenderkonzepten , 2016, Mensch & Computer.

[29]  Stefan Kopp,et al.  Co-constructing Grounded Symbols—Feedback and Incremental Adaptation in Human–Agent Dialogue , 2013, KI - Künstliche Intelligenz.

[30]  T. Bickmore,et al.  A Randomized Controlled Trial of an Automated Exercise Coach for Older Adults , 2013, Journal of the American Geriatrics Society.

[31]  A. Mihailidis,et al.  Difficulties in Automatic Speech Recognition of Dysarthric Speakers and Implications for Speech-Based Applications Used by the Elderly: A Literature Review , 2010, Assistive technology : the official journal of RESNA.

[32]  Stefan Kopp,et al.  Adaptive Grounding and Dialogue Management for Autonomous Conversational Assistants for Elderly Users , 2015, IVA.

[33]  Gregor Mehlmann,et al.  Modeling Grounding for Interactive Social Companions , 2015, KI - Künstliche Intelligenz.

[34]  Franz Kummert,et al.  Online nod detection in human-robot interaction , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[35]  Morteza Behrooz,et al.  Demonstration of an Always-On Companion for Isolated Older Adults , 2013, SIGDIAL Conference.