How to evaluate technologies for health behavior change in HCI research

New technologies for encouraging physical activity, healthy diet, and other types of health behavior change now frequently appear in the HCI literature. Yet, how such technologies should be evaluated within the context of HCI research remains unclear. In this paper, we argue that the obvious answer to this question - that evaluations should assess whether a technology brought about the intended change in behavior - is too limited. We propose that demonstrating behavior change is often infeasible as well as unnecessary for a meaningful contribution to HCI research, especially when in the early stages of design or when evaluating novel technologies. As an alternative, we suggest that HCI contributions should focus on efficacy evaluations that are tailored to the specific behavior-change intervention strategies (e.g., self-monitoring, conditioning) embodied in the system and studies that help gain a deep understanding of people's experiences with the technology.

[1]  R. Nelson.  Assessment and Therapeutic Functions of Self-Monitoring , 1977 .

[2]  M. Hersen,et al.  Progress in Behavior Modification , 1983 .

[3]  M. Csíkszentmihályi,et al.  Validity and Reliability of the Experience‐Sampling Method , 1987, The Journal of nervous and mental disease.

[4]  J. Kopp,et al.  Self-monitoring: A literature review of research and practice , 1988 .

[5]  K. Radtke.  Exercise Compliance in Cardiac Rehabilitation , 1989, Rehabilitation nursing : the official journal of the Association of Rehabilitation Nurses.

[6]  E. Higgins,et al.  Social psychology: Handbook of basic principles. , 1996 .

[7]  E. Higgins.  Knowledge activation: Accessibility, applicability, and salience. , 1996 .

[8]  C. Hoffman,et al.  Persons with chronic conditions. Their prevalence and costs. , 1996, JAMA.

[9]  Sara S Johnson,et al.  The transtheoretical model of behavior change. , 1998 .

[10]  R. Baumeister,et al.  Self-control as limited resource: regulatory depletion patterns. , 1998, Journal of personality and social psychology.

[11]  G. Marlatt,et al.  Relapse prevention and the maintenance of optimal health. , 1998 .

[12]  R. Baumeister,et al.  Self-regulation and depletion of limited resources: does self-control resemble a muscle? , 2000, Psychological bulletin.

[13]  W. Bickel,et al.  Reframing Health Behavior Change With Behavioral Economics , 2000 .

[14]  E. Deci,et al.  The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior , 2000 .

[15]  S. Blair,et al.  Physical activity behavior change: issues in adoption and maintenance. , 2000, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[16]  J. Bargh,et al.  The automated will: nonconscious activation and pursuit of behavioral goals. , 2001, Journal of personality and social psychology.

[17]  S. Gabriel,et al.  Systematic Review of the Literature , 2021, Adherence to Antiretroviral Therapy among Perinatal Women in Guyana.

[18]  B. J. Fogg,et al.  Persuasive technology: using computers to change what we think and do , 2002, UBIQ.

[19]  Mary A. Dolansky,et al.  Predictors of women's exercise maintenance after cardiac rehabilitation. , 2003, Journal of cardiopulmonary rehabilitation.

[20]  K. Witkiewitz,et al.  Relapse prevention for alcohol and drug problems: that was Zen, this is Tao. , 2004, The American psychologist.

[21]  Timothy W. Bickmore,et al.  Acceptance and usability of a relational agent interface by urban older adults , 2005, CHI Extended Abstracts.

[22]  Jeana Frost,et al.  Integrating glucometers and digital photography as experience capture tools to enhance patient understanding and communication of diabetes self-management practices , 2007, Personal and Ubiquitous Computing.

[23]  Silvia Lindtner,et al.  Fish'n'Steps: Encouraging Physical Activity with an Interactive Computer Game , 2006, UbiComp.

[24]  James A. Landay,et al.  Design requirements for technologies that encourage physical activity , 2006, CHI.

[25]  Dominique Brodbeck,et al.  Persuasiveness of a Mobile Lifestyle Coaching Application Using Social Facilitation , 2006, PERSUASIVE.

[26]  Kent Larson,et al.  Embedding Behavior Modification Strategies into a Consumer Electronic Device: A Case Study , 2006, UbiComp.

[27]  Robin Jeffries,et al.  CHI '06 Extended Abstracts on Human Factors in Computing Systems , 2006, CHI 2006.

[28]  William G. Griswold,et al.  Usability and Feasibility of PmEB: A Mobile Phone Application for Monitoring Real Time Caloric Balance , 2006, 2006 Pervasive Health Conference and Workshops.

[29]  D. Rothenbacher,et al.  Long term adherence to dietary recommendations after inpatient rehabilitation: prospective follow up study of patients with coronary heart disease , 2005, Heart.

[30]  Yvonne Rogers,et al.  Why It's Worth the Hassle: The Value of In-Situ Studies When Designing Ubicomp , 2007, UbiComp.

[31]  M. H. van den Berg,et al.  Internet-Based Physical Activity Interventions: A Systematic Review of the Literature , 2007, Journal of medical Internet research.

[32]  William G. Griswold,et al.  Usability and Feasibility of PmEB: A Mobile Phone Application for Monitoring Real Time Caloric Balance , 2006, Mob. Networks Appl..

[33]  James A. Landay,et al.  MyExperience: a system for in situ tracing and capturing of user feedback on mobile phones , 2007, MobiSys '07.

[34]  Matthew Chalmers,et al.  Shakra: Tracking and Sharing Daily Activity Levels with Unaugmented Mobile Phones , 2007, Mob. Networks Appl..

[35]  J. S. Sodhi,et al.  Using Internet and Mobile Phone Technology to Deliver an Automated Physical Activity Program: Randomized Controlled Trial , 2007, Journal of medical Internet research.

[36]  Louise Barkhuus,et al.  From Mice to Men - 24 Years of Evaluation in CHI , 2007, CHI 2007.

[37]  Ilkka Korhonen,et al.  Mobile Diary for Wellness Management—Results on Usage and Usability in Two User Studies , 2008, IEEE Transactions on Information Technology in Biomedicine.

[38]  David W. McDonald,et al.  Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays , 2008, UbiComp.

[39]  David W. McDonald,et al.  Activity sensing in the wild: a field trial of ubifit garden , 2008, CHI.

[40]  William Buxton,et al.  Usability evaluation considered harmful (some of the time) , 2008, CHI.

[41]  Lena Mamykina,et al.  MAHI: investigation of social scaffolding for reflective thinking in diabetes management , 2008, CHI.

[42]  Gaetano Borriello,et al.  BALANCE: towards a usable pervasive wellness application with accurate activity inference , 2009, HotMobile '09.

[43]  Wanda Pratt,et al.  Designing and remotely testing mobile diabetes video games , 2010, Journal of telemedicine and telecare.

[44]  Michael E. Labhard,et al.  Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness , 2010, Journal of medical Internet research.

[45]  Morgan Dixon,et al.  Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure , 2010, CHI.

[46]  Mladenka Tkalcic,et al.  Transtheoretical model of behavior change , 2011 .

[47]  Grady Booch,et al.  The Human Experience , 2012, IEEE Software.

[48]  K. Riekert,et al.  The handbook of health behavior change , 2013 .