From Reflection to Action: Combining Machine Learning with Expert Knowledge for Nutrition Goal Recommendations

Self-tracking can help personalize self-management interventions for chronic conditions like type 2 diabetes (T2D), but reflecting on personal data requires motivation and literacy. Machine learning (ML) methods can identify patterns, but a key challenge is making actionable suggestions based on personal health data. We introduce GlucoGoalie, which combines ML with an expert system to translate ML output into personalized nutrition goal suggestions for individuals with T2D. In a controlled experiment, participants with T2D found that goal suggestions were understandable and actionable. A 4-week in-the-wild deployment study showed that receiving goal suggestions augmented participants’ self-discovery, choosing goals highlighted the multifaceted nature of personal preferences, and the experience of following goals demonstrated the importance of feedback and context. However, we identified tensions between abstract goals and concrete eating experiences and found static text too ambiguous for complex concepts. We discuss implications for ML-based interventions and the need for systems that offer more interactivity, feedback, and negotiation.

[1]  H. Tighiouart,et al.  Estimating the reliability of glycemic index values and potential sources of methodological and biological variability. , 2016, The American journal of clinical nutrition.

[2]  David Elsweiler,et al.  Towards Automatic Meal Plan Recommendations for Balanced Nutrition , 2015, RecSys.

[3]  Bart Kamphorst,et al.  E-coaching systems - What they are, and what they aren't , 2017, Pers. Ubiquitous Comput..

[4]  T. Bodenheimer,et al.  Patient self-management of chronic disease in primary care. , 2002, JAMA.

[5]  Rebecca E. Grinter,et al.  EatWell: sharing nutrition-related memories in a low-income community , 2008, CSCW.

[6]  K. Cavanaugh.  Health literacy in diabetes care: explanation, evidence and equipment. , 2011, Diabetes management.

[7]  Deborah Estrin,et al.  Yum-Me: A Personalized Nutrient-Based Meal Recommender System , 2016, ACM Trans. Inf. Syst..

[8]  Phoebe Sengers,et al.  Fit4life: the design of a persuasive technology promoting healthy behavior and ideal weight , 2011, CHI.

[9]  4. Lifestyle Management: Standards of Medical Care in Diabetes—2018 , 2017, Diabetes Care.

[10]  Lawrence R. Ness,et al.  Are We There Yet - Data Saturation in Qualitative Research (TQR Published).pdf , 2015 .

[11]  T. Bickmore,et al.  A Randomized Controlled Trial of an Automated Exercise Coach for Older Adults , 2013, Journal of the American Geriatrics Society.

[12]  Sunmoo Yoon,et al.  Sometimes more is more: iterative participatory design of infographics for engagement of community members with varying levels of health literacy , 2016, J. Am. Medical Informatics Assoc..

[13]  Mary Czerwinski,et al.  Pocket Skills: A Conversational Mobile Web App To Support Dialectical Behavioral Therapy , 2018, CHI.

[14]  Peter C. Wright,et al.  Enhancing Personal Informatics Through Social Sensemaking , 2017, CHI.

[15]  Timothy W. Bickmore,et al.  A reusable framework for health counseling dialogue systems based on a behavioral medicine ontology , 2011, J. Biomed. Informatics.

[16]  Lars Vedel Kessing,et al.  MUBS: A Personalized Recommender System for Behavioral Activation in Mental Health , 2020, CHI.

[17]  K. Fitzpatrick,et al.  Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial , 2017, JMIR mental health.

[18]  Krzysztof Z. Gajos,et al.  Platemate: crowdsourcing nutritional analysis from food photographs , 2011, UIST.

[19]  Jeong-Whun Kim,et al.  "My Doctor is Keeping an Eye on Me!": Exploring the Clinical Applicability of a Mobile Food Logger , 2016, CHI.

[20]  Lior Rokach,et al.  Recommender Systems: Introduction and Challenges , 2015, Recommender Systems Handbook.

[21]  Jean-Claude Martin,et al.  WEnner: A Theoretically Motivated Approach for Tailored Coaching about Physical Activity , 2018, UbiComp/ISWC Adjunct.

[22]  Mark W. Newman,et al.  "My blood sugar is higher on the weekends": Finding a Role for Context and Context-Awareness in the Design of Health Self-Management Technology , 2019, CHI.

[23]  Steven K. Feiner,et al.  Leveraging Patient-Reported Outcomes Using Data Visualization , 2018, Applied Clinical Informatics.

[24]  Sayali S. Phatak,et al.  Adaptive step goals and rewards: a longitudinal growth model of daily steps for a smartphone-based walking intervention , 2018, Journal of Behavioral Medicine.

[25]  E. Segal,et al.  Personalized Nutrition by Prediction of Glycemic Responses , 2015, Cell.

[26]  Sean A. Munson,et al.  Taming data complexity in lifelogs: exploring visual cuts of personal informatics data , 2014, Conference on Designing Interactive Systems.

[27]  Lena Mamykina,et al.  A visual analytics approach for pattern-recognition in patient-generated data , 2018, J. Am. Medical Informatics Assoc..

[28]  Ambuj Tewari,et al.  An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventions , 2017, ArXiv.

[29]  C. Abraham,et al.  The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions , 2013, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[30]  Lena Mamykina,et al.  Personal Health Oracle: Explorations of Personalized Predictions in Diabetes Self-Management , 2019, CHI.

[31]  Joseph Jay Williams,et al.  SleepBandits: Guided Flexible Self-Experiments for Sleep , 2020, CHI.

[32]  Sean A. Munson,et al.  Exploring goal-setting, rewards, self-monitoring, and sharing to motivate physical activity , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[33]  Pawel W. Wozniak,et al.  Supporting Meaningful Personal Fitness: the Tracker Goal Evolution Model , 2018, CHI.

[34]  Qian Yang,et al.  Designing Theory-Driven User-Centric Explainable AI , 2019, CHI.

[35]  Rosalind W. Picard,et al.  Establishing the computer-patient working alliance in automated health behavior change interventions. , 2005, Patient education and counseling.

[36]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[37]  Jeffrey Dean,et al.  Scalable and accurate deep learning with electronic health records , 2018, npj Digital Medicine.

[38]  Lena Mamykina,et al.  Adapting the stage-based model of personal informatics for low-resource communities in the context of type 2 diabetes , 2020, J. Biomed. Informatics.

[39]  Lena Mamykina,et al.  Collective Sensemaking in Online Health Forums , 2015, CHI.

[40]  Elizabeth Greenberg,et al.  A First Look at the Literacy of America's Adults in the 21st Century. NCES 2006-470. , 2006 .

[41]  Bernd Ludwig,et al.  Engendering Health with Recommender Systems , 2016, RecSys.

[42]  Matthew Kay,et al.  A Patient-Centered Proposal for Bayesian Analysis of Self-Experiments for Health , 2019, J. Heal. Informatics Res..

[43]  Lena Mamykina,et al.  Data-driven health management: reasoning about personally generated data in diabetes with information technologies , 2016, J. Am. Medical Informatics Assoc..

[44]  Hanna Schäfer,et al.  Personalized Support for Healthy Nutrition Decisions , 2016, RecSys.

[45]  Daniel E. Rivera,et al.  Development of a Control-Oriented Model of Social Cognitive Theory for Optimized mHealth Behavioral Interventions , 2020, IEEE Transactions on Control Systems Technology.

[46]  Sue Sing Lim,et al.  Choose Your Foods, Food Lists for Diabetes. , 2015 .

[47]  Lena Mamykina,et al.  MAHI: investigation of social scaffolding for reflective thinking in diabetes management , 2008, CHI.

[48]  Ben Jelen,et al.  Evaluation of a Food Portion Size Estimation Interface for a Varying Literacy Population , 2016, CHI.

[49]  David W. McDonald,et al.  Goal-setting considerations for persuasive technologies that encourage physical activity , 2009, Persuasive '09.

[50]  Lena Mamykina,et al.  Mechanistic machine learning: how data assimilation leverages physiologic knowledge using Bayesian inference to forecast the future, infer the present, and phenotype , 2018, J. Am. Medical Informatics Assoc..

[51]  Deborah Estrin,et al.  Small data, where n = me , 2014, Commun. ACM.

[52]  Mi Zhang,et al.  MyBehavior: automatic personalized health feedback from user behaviors and preferences using smartphones , 2015, UbiComp.

[53]  Sean A Munson,et al.  Identifying and Planning for Individualized Change , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[54]  Jodi Forlizzi,et al.  A stage-based model of personal informatics systems , 2010, CHI.

[55]  Joan Marie Culley,et al.  Exploring app features with outcomes in mHealth studies involving chronic respiratory diseases, diabetes, and hypertension: a targeted exploration of the literature , 2018, J. Am. Medical Informatics Assoc..

[56]  Chris Shaw,et al.  Towards developing an e-coach to support arthritis patients in maintaining a physically active lifestyle , 2018, PervasiveHealth.

[57]  Lena Mamykina,et al.  Enabling Personalized Decision Support with Patient-Generated Data and Attributable Components. , 2020, Journal of biomedical informatics.

[58]  Jessica S. Ancker,et al.  Good intentions are not enough: how informatics interventions can worsen inequality , 2018, J. Am. Medical Informatics Assoc..

[59]  Jodi Forlizzi,et al.  Understanding my data, myself: supporting self-reflection with ubicomp technologies , 2011, UbiComp '11.

[60]  Sean A. Munson,et al.  TummyTrials: A Feasibility Study of Using Self-Experimentation to Detect Individualized Food Triggers , 2017, CHI.

[61]  F. Collins,et al.  A new initiative on precision medicine. , 2015, The New England journal of medicine.

[62]  P. Aschner,et al.  New IDF clinical practice recommendations for managing type 2 diabetes in primary care. , 2017, Diabetes research and clinical practice.

[63]  Esteban G. Tabak,et al.  Explanation of Variability and Removal of Confounding Factors from Data through Optimal Transport , 2018 .

[64]  Lena Mamykina,et al.  No longer wearing: investigating the abandonment of personal health-tracking technologies on craigslist , 2015, UbiComp.

[65]  Sean A. Munson,et al.  Crumbs: Lightweight Daily Food Challenges to Promote Engagement and Mindfulness , 2016, CHI.

[66]  WhittakerSteve,et al.  What Does All This Data Mean for My Future Mood? Actionable Analytics and Targeted Reflection for Emotional Well-Being , 2017 .

[67]  Y. Jang,et al.  Standards of Medical Care in Diabetes-2010 by the American Diabetes Association: Prevention and Management of Cardiovascular Disease , 2010 .

[68]  Christian Koehler,et al.  Why we use and abandon smart devices , 2015, UbiComp.

[69]  Edward H. Wagner,et al.  Eliciting Values of Patients with Multiple Chronic Conditions: Evaluation of a Patient-centered Framework , 2017, AMIA.

[70]  Sean A. Munson,et al.  A framework for self-experimentation in personalized health , 2016, J. Am. Medical Informatics Assoc..

[71]  Kirsten Swearingen,et al.  Beyond Algorithms: An HCI Perspective on Recommender Systems , 2001 .

[72]  Christoph Trattner,et al.  Towards Health (Aware) Recommender Systems , 2017, DH.

[73]  Jacob Solomon,et al.  Graphics help patients distinguish between urgent and non-urgent deviations in laboratory test results , 2016, J. Am. Medical Informatics Assoc..

[74]  The Diabetes Prevention Program (DPP): description of lifestyle intervention. , 2002, Diabetes care.

[75]  Eric B. Hekler,et al.  Helping Users Set Rules for Defining Short-Term Activity Goals , 2016, CHI Extended Abstracts.

[76]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[77]  Wanda Pratt,et al.  Understanding quantified-selfers' practices in collecting and exploring personal data , 2014, CHI.

[78]  D. Gustafson,et al.  Predictive modeling of addiction lapses in a mobile health application. , 2014, Journal of substance abuse treatment.

[79]  Chen Wang,et al.  AllergyBot: A Chatbot Technology Intervention for Young Adults with Food Allergies Dining Out , 2017, CHI Extended Abstracts.

[80]  E. Coiera,et al.  Design and Implementation of Behavioral Informatics Interventions , 2017 .

[81]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[82]  James Fogarty,et al.  Rethinking the Mobile Food Journal: Exploring Opportunities for Lightweight Photo-Based Capture , 2015, CHI.

[83]  Laura R. Saslow,et al.  Nutrition Therapy for Adults With Diabetes or Prediabetes: A Consensus Report , 2019, Diabetes Care.

[84]  Wanda Pratt,et al.  How to evaluate technologies for health behavior change in HCI research , 2011, CHI.

[85]  Lena Mamykina,et al.  Participatory approach to the development of a knowledge base for problem-solving in diabetes self-management , 2016, Int. J. Medical Informatics.

[86]  M. Peek,et al.  Diabetes Health Disparities , 2007, Medical care research and review : MCRR.

[87]  K. Ohe,et al.  Validating the use of photos to measure dietary intake: the method used by DialBetics, a smartphone-based self-management system for diabetes patients , 2016, Diabetology International.

[88]  Krzysztof Z. Gajos,et al.  The Role of Explanations in Casual Observational Learning about Nutrition , 2017, CHI.

[89]  Wanda Pratt,et al.  Healthcare in the pocket: Mapping the space of mobile-phone health interventions , 2012, J. Biomed. Informatics.

[90]  Helena M. Mentis,et al.  Turning to Peers: Integrating Understanding of the Self, the Condition, and Others’ Experiences in Making Sense of Complex Chronic Conditions , 2016, Computer Supported Cooperative Work (CSCW).

[91]  Laura Johnson,et al.  How Many Interviews Are Enough? , 2006 .

[92]  Rebecca E. Grinter,et al.  Let's play!: mobile health games for adults , 2010, UbiComp.

[93]  Konrad Tollmar,et al.  Health Mashups: Presenting Statistical Patterns between Wellbeing Data and Context in Natural Language to Promote Behavior Change , 2013, TCHI.

[94]  Sean A. Munson,et al.  Examining Opportunities for Goal-Directed Self-Tracking to Support Chronic Condition Management , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..