Understanding How Users Experience the Physiological Expression of Non-humanoid Voice-based Conversational Agent in Healthcare Services

Interactions with voice-based conversational agents (VCAs) in non-humanoid forms are becoming increasingly pervasive, and researches on non-humanoid VCAs engaging diverse human traits have been conducted. However, there have never been studies employing living body's physiological states to be expressed solely through the voice of such VCAs yet. As physiological expressions of such VCAs can have potential for manifesting health-related issues in a human-like way, we selected healthcare scenarios as a case for exploring novel user experiences that they can induce. We conducted design workshops for identifying design considerations and design opportunities for the physiologically expressible VCAs in the healthcare service domain. Following these findings, we designed the new concept of physiologically expressible healthcare VCAs and conducted a Wizard-of-Oz user study. Finally, we summarize the unique user experiences on physiologically expressible VCA's healthcare services and user perceptions of its physiological expressions, and discuss design implications for physiologically expressible VCAs.

[1]  J. Ayers,et al.  Responses to addiction help-seeking from Alexa, Siri, Google Assistant, Cortana, and Bixby intelligent virtual assistants , 2020, npj Digital Medicine.

[2]  Jessica A. Chen,et al.  Conversational agents in healthcare: a systematic review , 2018, J. Am. Medical Informatics Assoc..

[3]  Mats Hagberg,et al.  Texting on mobile phones and musculoskeletal disorders in young adults: A five-year cohort study. , 2017, Applied ergonomics.

[4]  T. Shibata,et al.  Robot Therapy: A New Approach for Mental Healthcare of the Elderly – A Mini-Review , 2010, Gerontology.

[5]  Stephenie C. Lemon,et al.  The influence of family, friend, and coworker social support and social undermining on weight gain prevention among adults , 2014, Obesity.

[6]  Walter Brenner,et al.  Design Thinking as Mindset, Process, and Toolbox , 2016 .

[7]  R. Wing,et al.  Benefits of recruiting participants with friends and increasing social support for weight loss and maintenance. , 1999, Journal of consulting and clinical psychology.

[8]  Farah Magrabi,et al.  Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures , 2020, Journal of medical Internet research.

[9]  Adam S. Miner,et al.  Smartphone-Based Conversational Agents and Responses to Questions About Mental Health, Interpersonal Violence, and Physical Health. , 2016, JAMA internal medicine.

[10]  J. Dale,et al.  What is the effect of peer support on diabetes outcomes in adults? A systematic review , 2012, Diabetic medicine : a journal of the British Diabetic Association.

[11]  Benjamin R. Cowan,et al.  What Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents , 2019, CHI.

[12]  Todd Kulesza,et al.  Tell me more?: the effects of mental model soundness on personalizing an intelligent agent , 2012, CHI.

[13]  C. B. Taylor,et al.  Social Support for Healthy Behaviors: Scale Psychometrics and Prediction of Weight Loss Among Women in a Behavioral Program , 2012, Obesity.

[14]  Anita Burgun-Parenthoine,et al.  Automatic concept extraction from spoken medical reports , 2003, Int. J. Medical Informatics.

[15]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[16]  Florian Alt,et al.  At Your Service: Designing Voice Assistant Personalities to Improve Automotive User Interfaces , 2019, CHI.

[17]  Lola Cañamero,et al.  Making New “New AI” Friends: Designing a Social Robot for Diabetic Children from an Embodied AI Perspective , 2016, Int. J. Soc. Robotics.

[18]  Arne Jönsson,et al.  Wizard of Oz studies: why and how , 1993, IUI '93.

[19]  Andrés Montoyo,et al.  Advances on natural language processing , 2007, Data Knowl. Eng..

[20]  Takanori Shibata,et al.  Psychological and Social Effects of One Year Robot Assisted Activity on Elderly People at a Health Service Facility for the Aged , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[21]  Sangsu Lee,et al.  "What does your Agent look like?": A Drawing Study to Understand Users' Perceived Persona of Conversational Agent , 2019, CHI Extended Abstracts.

[22]  S. Preum,et al.  A Review of Cognitive Assistants for Healthcare , 2021, ACM Comput. Surv..

[23]  Marta Perez Garcia,et al.  Exploring the Uncanny Valley Theory in the Constructs of a Virtual Assistant Personality , 2019, IntelliSys.

[24]  Wilhelm Hasselbring,et al.  The SI challenge in health care , 2000, CACM.

[25]  Clifford Nass,et al.  Does computer-generated speech manifest personality? an experimental test of similarity-attraction , 2000, CHI.

[26]  E. Alagha,et al.  Evaluating the quality of voice assistants’ responses to consumer health questions about vaccines: an exploratory comparison of Alexa, Google Assistant and Siri , 2019, BMJ Health & Care Informatics.

[27]  Khalil Sima'an,et al.  Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship , 2006, Computational Linguistics.

[28]  Rosemary Stockdale,et al.  Peer-to-peer online communities for people with chronic diseases: a conceptual framework , 2008, J. Syst. Inf. Technol..

[29]  Chinmay Kulkarni,et al.  One Voice Fits All? , 2019, Proc. ACM Hum. Comput. Interact..

[30]  Xiao Li,et al.  Machine Learning Paradigms for Speech Recognition: An Overview , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[31]  Matthias Rauterberg,et al.  Electrodermal activity: Explorations in the psychophysiology of engagement with social robots in dementia , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[32]  Sarah Sharples,et al.  Voice Interfaces in Everyday Life , 2018, CHI.

[33]  C. Judd,et al.  What the Voice Reveals: Within- and Between-Category Stereotyping on the Basis of Voice , 2006, Personality & social psychology bulletin.

[34]  Jenny Preece,et al.  Online Communities: Designing Usability and Supporting Sociability , 2000 .

[35]  J. Cacioppo,et al.  On seeing human: a three-factor theory of anthropomorphism. , 2007, Psychological review.

[36]  Clifford Nass,et al.  The Effects of Emotion of Voice in Synthesized and Recorded Speech , 2001 .

[37]  Eva Brandt,et al.  Evoking the future: Drama and props in user centered design , 2000 .

[38]  M. Funnell.  Peer-based behavioural strategies to improve chronic disease self-management and clinical outcomes: evidence, logistics, evaluation considerations and needs for future research. , 2010, Family practice.

[39]  Judgments of Emotion by Nurses and Students Given Double-Bind Information on a Patient's Tone of Voice and Message Content , 2000, Perceptual and motor skills.

[40]  Anselm L. Strauss,et al.  Qualitative Analysis For Social Scientists , 1987 .

[41]  Jen Ross,et al.  The 'digital native' and 'digital immigrant': a dangerous opposition , 2007 .

[42]  Youn-Kyung Lim,et al.  Co-Performing Agent: Design for Building User-Agent Partnership in Learning and Adaptive Services , 2019, CHI.

[43]  Youn-kyung Lim,et al.  Developing a Design Guide for Consistent Manifestation of Conversational Agent Personalities , 2019 .

[44]  Bijan Najafi,et al.  Health Sensors, Smart Home Devices, and the Internet of Medical Things: An Opportunity for Dramatic Improvement in Care for the Lower Extremity Complications of Diabetes , 2018, Journal of diabetes science and technology.

[45]  John Torous,et al.  The efficacy of smartphone‐based mental health interventions for depressive symptoms: a meta‐analysis of randomized controlled trials , 2017, World psychiatry : official journal of the World Psychiatric Association.

[46]  C. Nass,et al.  Social-Psychological Origins of Feelings of Presence: Creating Social Presence With Machine-Generated Voices , 2005 .

[47]  Atieh Poushneh.  Humanizing voice assistant: The impact of voice assistant personality on consumers’ attitudes and behaviors , 2021 .

[48]  Elizabeth Sillence,et al.  Modeling patient engagement in peer-to-peer healthcare , 2014, WWW.

[49]  Eric Dishman,et al.  ACTORS,HAIRDOS& VIDEOTAPE= INFORMANCE DESIGN Usingperformancetechniquesin multi-disciplinaryobservationbaseddesign , 1994 .

[50]  Joel J. P. C. Rodrigues,et al.  Home-based exercise system for patients using IoT enabled smart speaker , 2017, 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom).

[51]  C P Bradley,et al.  Giving voice to the lifeworld. More humane, more effective medical care? A qualitative study of doctor-patient communication in general practice. , 2001, Social science & medicine.

[52]  George TERZOPOULOS,et al.  Voice Assistants and Smart Speakers in Everyday Life and in Education , 2020, Informatics Educ..