Personalizing Mobile Fitness Apps using Reinforcement Learning

Despite the vast number of mobile fitness applications (apps) and their potential advantages in promoting physical activity, many existing apps lack behavior-change features and are not able to maintain behavior change motivation. This paper describes a novel fitness app called CalFit, which implements important behavior-change features like dynamic goal setting and self-monitoring. CalFit uses a reinforcement learning algorithm to generate personalized daily step goals that are challenging but attainable. We conducted the Mobile Student Activity Reinforcement (mSTAR) study with 13 college students to evaluate the efficacy of the CalFit app. The control group (receiving goals of 10,000 steps/day) had a decrease in daily step count of 1,520 (SD ± 740) between baseline and 10-weeks, compared to an increase of 700 (SD ± 830) in the intervention group (receiving personalized step goals). The difference in daily steps between the two groups was 2,220, with a statistically significant p = 0.039.

[1]  N. Barengo,et al.  Weight loss by mobile phone: a 1-year effectiveness study , 2009, Public Health Nutrition.

[2]  Anca D. Dragan,et al.  Cooperative Inverse Reinforcement Learning , 2016, NIPS.

[3]  W. Riley,et al.  Evaluation of a PDA-based Dietary Assessment and Intervention Program: A Randomized Controlled Trial , 2006, Journal of the American College of Nutrition.

[4]  Judith Masthoff,et al.  Personalising Healthy Eating Messages to Age, Gender and Personality: Using Cialdini's Principles and Framing , 2017, IUI Companion.

[5]  Youri van Pinxteren,et al.  Deriving a recipe similarity measure for recommending healthful meals , 2011, IUI '11.

[6]  Brianna S Fjeldsoe,et al.  MobileMums: A Randomized Controlled Trial of an SMS-Based Physical Activity Intervention , 2010, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[7]  N. Joo,et al.  Mobile phone short message service messaging for behaviour modification in a community-based weight control programme in Korea , 2007, Journal of telemedicine and telecare.

[8]  Sean A. Munson,et al.  Exploring goal-setting, rewards, self-monitoring, and sharing to motivate physical activity , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[9]  Anil Aswani,et al.  Behavioral modeling in weight loss interventions , 2019, Eur. J. Oper. Res..

[10]  Christopher D Gardner,et al.  Using hand-held computer technologies to improve dietary intake. , 2008, American Journal of Preventive Medicine.

[11]  M. Marcus,et al.  Effect of Wearable Technology Combined With a Lifestyle Intervention on Long-term Weight Loss: The IDEA Randomized Clinical Trial. , 2016, JAMA.

[13]  Kevin L. Joiner,et al.  A Novel Diabetes Prevention Intervention Using a Mobile App: A Randomized Controlled Trial With Overweight Adults at Risk. , 2015, American journal of preventive medicine.

[14]  Audie A Atienza,et al.  Promoting physical activity through hand-held computer technology. , 2008, American journal of preventive medicine.

[15]  Yuichi Fujiki,et al.  iPhone as a physical activity measurement platform , 2010, CHI Extended Abstracts.

[16]  Karon E. MacLean,et al.  Real-time gait classification for persuasive smartphone apps: structuring the literature and pushing the limits , 2013, IUI '13.

[17]  M. Adams,et al.  An Adaptive Physical Activity Intervention for Overweight Adults: A Randomized Controlled Trial , 2013, PloS one.

[18]  A. Bandura.  Social cognitive theory of moral thought and action. , 1991 .

[19]  K. Volpp,et al.  Accuracy of smartphone applications and wearable devices for tracking physical activity data. , 2015, JAMA.

[20]  Zuo-Jun Max Shen,et al.  Inverse Optimization with Noisy Data , 2015, Oper. Res..

[21]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[22]  Anil Aswani,et al.  Behavioral analytics for myopic agents , 2017, Eur. J. Oper. Res..

[23]  John G. Breslin,et al.  Fitness—There's an App for That: Review of Mobile Fitness Apps , 2013 .

[24]  Dennis Wiebusch,et al.  Remind Me: An Adaptive Recommendation-Based Simulation of Biographic Associations , 2016, IUI.

[25]  S. Bauer,et al.  Enhancement of care through self-monitoring and tailored feedback via text messaging and their use in the treatment of childhood overweight. , 2010, Patient education and counseling.

[26]  Lauren A. Grieco,et al.  Validation of Physical Activity Tracking via Android Smartphones Compared to ActiGraph Accelerometer: Laboratory-Based and Free-Living Validation Studies , 2015, JMIR mHealth and uHealth.

[27]  J. Keysor,et al.  Does late-life physical activity or exercise prevent or minimize disablement? A critical review of the scientific evidence. , 2003, American journal of preventive medicine.

[28]  Charles Abraham,et al.  A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management , 2016, International Journal of Behavioral Nutrition and Physical Activity.

[29]  S. Sears,et al.  Critical Analysis , 2014 .

[30]  W. Nilsen,et al.  Health behavior models in the age of mobile interventions: are our theories up to the task? , 2011, Translational behavioral medicine.

[31]  A. Sanabria,et al.  Randomized controlled trial. , 2005, World journal of surgery.

[32]  M. Sevick,et al.  The Effect of Electronic Self‐Monitoring on Weight Loss and Dietary Intake: A Randomized Behavioral Weight Loss Trial , 2011, Obesity.

[33]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[34]  K. Patrick,et al.  A Text Message–Based Intervention for Weight Loss: Randomized Controlled Trial , 2009, Journal of medical Internet research.

[35]  J. Knight,et al.  Physical inactivity: associated diseases and disorders. , 2012, Annals of clinical and laboratory science.

[36]  E. A. Locke,et al.  Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. , 2002, The American psychologist.

[37]  Steve Whittaker,et al.  Finding the Adaptive Sweet Spot: Balancing Compliance and Achievement in Automated Stress Reduction , 2015, CHI.

[38]  Sanjay Krishnan,et al.  HIRL: Hierarchical Inverse Reinforcement Learning for Long-Horizon Tasks with Delayed Rewards , 2016, ArXiv.

[39]  Lora E Burke,et al.  Mobile applications for weight management: theory-based content analysis. , 2013, American journal of preventive medicine.

[40]  Lora Giangregorio,et al.  Behavior Change Techniques Present in Wearable Activity Trackers: A Critical Analysis , 2016, JMIR mHealth and uHealth.

[41]  Youngseok Lee,et al.  A Look at Wearable Abandonment , 2017, 2017 18th IEEE International Conference on Mobile Data Management (MDM).

[42]  E. McAuley,et al.  Self‐Efficacy Determinants and Consequences of Physical Activity , 2000, Exercise and sport sciences reviews.

[43]  J. S. Sodhi,et al.  Using Internet and Mobile Phone Technology to Deliver an Automated Physical Activity Program: Randomized Controlled Trial , 2007, Journal of medical Internet research.

[44]  E. Vittinghoff,et al.  Innovation to motivation--pilot study of a mobile phone intervention to increase physical activity among sedentary women. , 2010, Preventive medicine.

[45]  J. Leskovec,et al.  Large-scale physical activity data reveal worldwide activity inequality , 2017, Nature.

[46]  J. Stoker,et al.  The Department of Health and Human Services. , 1999, Home healthcare nurse.

[47]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[48]  David W. McDonald,et al.  Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays , 2008, UbiComp.

[49]  J. Annesi,et al.  Goal-Setting Protocol in Adherence to Exercise by Italian Adults , 2002, Perceptual and motor skills.