Designing an Adaptive Embodied Conversational Agent for Health Literacy: a User Study

Access to healthcare advice is crucial to promote healthy societies. Many factors shape how access might be constrained, such as economic status, education or, as the COVID-19 pandemic has shown, remote consultations with health practitioners. Our work focuses on providing pre/post-natal advice to maternal women. A salient factor of our work concerns the design and deployment of embodied conversation agents (ECAs) which can sense the (health) literacy of users and adapt to scaffold user engagement in this setting. We present an account of a Wizard of Oz user study of 'ALTCAI', an ECA with three modes of interaction (i.e., adaptive speech and text, adaptive ECA, and non-adaptive ECA). We compare reported engagement with these modes from 44 maternal women who have differing levels of literacy. The study shows that a combination of embodiment and adaptivity scaffolds reported engagement, but matters of health-literacy and language introduce nuanced considerations for the design of ECAs.

[1]  Russell Beale,et al.  The Impact of an Embodied Agent's Emotional Expressions Over Multiple Interactions , 2015, Interact. Comput..

[2]  C. Sidner,et al.  Automated interventions for multiple health behaviors using conversational agents. , 2013, Patient education and counseling.

[3]  Mark O. Riedl,et al.  The Visual Design and Implementation of an Embodied Conversational Agent in a Shared Decision-Making Context (eCoach) , 2015, HCI.

[4]  J. V. van Weert,et al.  Low Health Literacy and Evaluation of Online Health Information: A Systematic Review of the Literature , 2015, Journal of Medical Internet Research.

[5]  J. Mojoyinola.  Influence of Maternal Health Literacy on Healthy Pregnancy and Pregnancy Outcomes of Women Attending Public Hospitals in Ibadan, Oyo State, Nigeria , 2011 .

[6]  Martin H. Luerssen,et al.  Virtual Agents as a Service: Applications in Healthcare , 2018, IVA.

[7]  Naphtali Rishe,et al.  I Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health Interventions , 2013, TMIS.

[8]  Timothy Bickmore,et al.  Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents , 2009, CHI.

[9]  T. Jong.  Cognitive load theory, educational research, and instructional design: some food for thought , 2010 .

[10]  Paul A. Cairns,et al.  A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form , 2018, Int. J. Hum. Comput. Stud..

[11]  Peter Robinson,et al.  Cross-dataset learning and person-specific normalisation for automatic Action Unit detection , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[12]  T. Kanda,et al.  Psychology in human-robot communication: an attempt through investigation of negative attitudes and anxiety toward robots , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[13]  Jessica A. Chen,et al.  Conversational agents in healthcare: a systematic review , 2018, J. Am. Medical Informatics Assoc..

[14]  Chris Greenhalgh,et al.  Virtual Human Questionnaire for Analysis of Depression, Anxiety and Personality , 2019, IVA.

[15]  Timothy Bickmore,et al.  Use of an Interactive Computer Agent to Support Breastfeeding , 2013, Maternal and Child Health Journal.

[16]  David A. Kindig,et al.  Health literacy : a prescription to end confusion , 2004 .

[17]  Peggy W. Murphy,et al.  Rapid Estimate of Adult Literacy in Medicine (REALM): A Quick Reading Test for Patients. , 1993 .

[18]  Hani Hagras,et al.  Toward Human-Understandable, Explainable AI , 2018, Computer.

[19]  Laura M. Pfeifer,et al.  Using computer agents to explain medical documents to patients with low health literacy. , 2009, Patient education and counseling.

[20]  N. Bendycki.  Health literacy. , 2008, Marketing health services.

[21]  Jenny X. Liu,et al.  A Systematic Review of the Role of Proprietary and Patent Medicine Vendors in Healthcare Provision in Nigeria , 2015, PloS one.

[22]  M. Neerincx,et al.  How should a virtual agent present psychoeducation? Influence of verbal and textual presentation on adherence , 2017, Technology and health care : official journal of the European Society for Engineering and Medicine.

[23]  Ronald Rosenfeld,et al.  HealthLine: Speech-based access to health information by low-literate users , 2007, 2007 International Conference on Information and Communication Technologies and Development.

[24]  Martin Porcheron,et al.  Pulling Back the Curtain on the Wizards of Oz , 2021, Proc. ACM Hum. Comput. Interact..

[25]  R. Abidoye,et al.  The relationship of poverty on malnourished children in Lagos, Nigeria , 1999 .

[26]  Catherine Pelachaud,et al.  From brows to trust: evaluating embodied conversational agents , 2004 .

[27]  Timothy W. Bickmore,et al.  Usability of Conversational Agents by Patients with Inadequate Health Literacy: Evidence from Two Clinical Trials , 2010, Journal of health communication.

[28]  Hermie Hermens,et al.  Design Features of Embodied Conversational Agents in eHealth: a Literature Review , 2020, Int. J. Hum. Comput. Stud..

[29]  Kentaro Toyama,et al.  Designing mobile interfaces for novice and low-literacy users , 2011, TCHI.

[30]  Raja Parasuraman,et al.  A Little Anthropomorphism Goes a Long Way , 2017, Hum. Factors.

[31]  R. Dowse,et al.  Applicability of the REALM health literacy test to an English second-language South African population , 2010, Pharmacy World & Science.

[32]  مسعود رسول آبادی,et al.  2011 , 2012, The Winning Cars of the Indianapolis 500.

[33]  Timothy W. Bickmore,et al.  Looking the Part: The Effect of Attire and Setting on Perceptions of a Virtual Health Counselor , 2018, IVA.

[34]  Louis-Philippe Morency,et al.  OpenFace 2.0: Facial Behavior Analysis Toolkit , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[35]  T. Marteau,et al.  Development of a measure of informed choice suitable for use in low literacy populations. , 2007, Patient education and counseling.

[36]  S. Kripalani,et al.  Patient Literacy and Question-asking Behavior During the Medical Encounter: A Mixed-methods Analysis , 2007, Journal of General Internal Medicine.