Spoiled for Choice? Personalized Recommendation for Healthcare Decisions: A Multi-Armed Bandit Approach with a Dynamic Discrete-Choice Scheme

Online healthcare communities provide users with various healthcare interventions to promote healthy behavior and improve adherence. When faced with too many intervention choices, however, individuals may find it difficult to decide which option to take, especially when they lack the experience or knowledge to evaluate different options. The choice overload issue may negatively affect users' engagement in health management. In this study, we take a design-science perspective to propose a recommendation framework that helps users to select healthcare interventions. Taking into account that users' health behaviors can be highly dynamic and diverse, we propose a multi-armed bandit (MAB)-driven recommendation framework, which enables us to adaptively learn users' preference variations while promoting recommendation diversity in the meantime. To better adapt an MAB to the healthcare context, we synthesize two innovative model components based on prominent health theories. The first component is a deep-learning-based feature engineering procedure, which is designed to learn crucial recommendation contexts in regard to users' sequential health histories, health-management experiences, preferences, and intrinsic attributes of healthcare interventions. The second component is a diversity constraint, which structurally diversifies recommendations in different dimensions to provide users with well-rounded support. We apply our approach to an online weight management context and evaluate it rigorously through a series of experiments. Our results demonstrate that each of the design components is effective and that our recommendation design outperforms a wide range of state-of-the-art recommendation systems. Our study contributes to the research on the application of business intelligence and has implications for multiple stakeholders, including online healthcare platforms, policymakers, and users.

[1]  Peter S. Fader,et al.  Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments , 2016, Mark. Sci..

[2]  A. Bandura.  Health promotion from the perspective of social cognitive theory , 1998 .

[3]  Eli Pariser,et al.  The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think , 2012 .

[4]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[5]  Ram D. Gopal,et al.  Empirical Analysis of the Impact of Recommender Systems on Sales , 2010, J. Manag. Inf. Syst..

[6]  Jean Harvey-Berino,et al.  Internet-based weight control: the relationship between web features and weight loss. , 2008, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[7]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[8]  Annie Chen,et al.  Context-Aware Collaborative Filtering System: Predicting the User's Preference in the Ubiquitous Computing Environment , 2005, LoCA.

[9]  Kristjan H. Greenewald,et al.  Action Centered Contextual Bandits , 2017, NIPS.

[10]  M. Franz,et al.  The Answer to Weight Loss Is Easy—Doing It Is Hard! , 2001 .

[11]  M. Dimatteo,et al.  Social support and patient adherence to medical treatment: a meta-analysis. , 2004, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[12]  J. Wardle,et al.  The association between weight loss and engagement with a web-based food and exercise diary in a commercial weight loss programme: a retrospective analysis , 2011, The international journal of behavioral nutrition and physical activity.

[13]  Ben R. Newell,et al.  Unpacking the Exploration–Exploitation Tradeoff: A Synthesis of Human and Animal Literatures , 2015 .

[14]  Maarten Speekenbrink,et al.  Uncertainty and Exploration in a Restless Bandit Problem , 2015, Top. Cogn. Sci..

[15]  Gillian King,et al.  Social Support Processes and the Adaptation of Individuals With Chronic Disabilities , 2006, Qualitative health research.

[16]  Lihong Li,et al.  An Empirical Evaluation of Thompson Sampling , 2011, NIPS.

[17]  Mária Bieliková,et al.  Effective hierarchical vector-based news representation for personalized recommendation , 2012, Comput. Sci. Inf. Syst..

[18]  David Buttler,et al.  Tracking multiple topics for finding interesting articles , 2007, KDD '07.

[19]  Qiong Wu,et al.  Diversity-Promoting Deep Reinforcement Learning for Interactive Recommendation , 2019, ICCSE.

[20]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[21]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[22]  Flemming Topsøe,et al.  Jensen-Shannon divergence and Hilbert space embedding , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[23]  O. Ogbeiwi.  General concepts of goals and goal-setting in healthcare: A narrative review , 2018, Journal of Management & Organization.

[24]  John Riedl,et al.  Recommender systems: from algorithms to user experience , 2012, User Modeling and User-Adapted Interaction.

[25]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[26]  A. Bandura.  Social cognitive theory of self-regulation☆ , 1991 .

[27]  Bernd Ludwig,et al.  Matrix factorization techniques for context aware recommendation , 2011, RecSys '11.

[28]  Roger H. L. Chiang,et al.  Big Data Research in Information Systems: Toward an Inclusive Research Agenda , 2016, J. Assoc. Inf. Syst..

[29]  L. Yan.  Good Intentions, Bad Outcomes: The Effects of Mismatches in Social Support and Health Outcomes in an Online Weight Loss Community , 2015 .

[30]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[31]  K. Hall,et al.  Maintenance of Lost Weight and Long-Term Management of Obesity. , 2018, The Medical clinics of North America.

[32]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[33]  J. Hebebrand,et al.  [Obesity and overweight]. , 2009, Zeitschrift fur Kinder- und Jugendpsychiatrie und Psychotherapie.

[34]  S. Shumaker,et al.  Toward a Theory of Social Support: Closing Conceptual Gaps , 1984 .

[35]  E. A. Locke,et al.  A theory of goal setting & task performance , 1990 .

[36]  John M. Levine,et al.  To stay or leave?: the relationship of emotional and informational support to commitment in online health support groups , 2012, CSCW.

[37]  Kurt C Stange,et al.  Transforming physician practices to patient-centered medical homes: lessons from the national demonstration project. , 2011, Health affairs.

[38]  Liang Tang,et al.  Ensemble contextual bandits for personalized recommendation , 2014, RecSys '14.

[39]  Dominik Endres,et al.  A new metric for probability distributions , 2003, IEEE Transactions on Information Theory.

[40]  John Langford,et al.  Doubly Robust Policy Evaluation and Learning , 2011, ICML.

[41]  Xiaoyan Zhu,et al.  Contextual Combinatorial Bandit and its Application on Diversified Online Recommendation , 2014, SDM.

[42]  Param Vir Singh,et al.  A Hidden Markov Model for Collaborative Filtering , 2010, MIS Q..

[43]  Kartik Hosanagar,et al.  Blockbuster Culture's Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity , 2007, Manag. Sci..

[44]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[45]  Bahador Bahrami,et al.  Stochastic satisficing account of confidence in uncertain value-based decisions , 2018, PloS one.

[46]  F. O. Isinkaye,et al.  Recommendation systems: Principles, methods and evaluation , 2015 .

[47]  Veda C. Storey,et al.  Business Intelligence and Analytics: From Big Data to Big Impact , 2012, MIS Q..

[48]  K. Margaritis,et al.  Analysis of Recommender Systems’ Algorithms , 2003 .

[49]  Hal Daumé,et al.  Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback , 2017, EMNLP.

[50]  Yi Tay,et al.  Deep Learning based Recommender System: A Survey and New Perspectives , 2018 .

[51]  Osahon Ogbeiwi,et al.  Why written objectives need to be really SMART , 2017 .

[52]  Antti Oulasvirta,et al.  When more is less: the paradox of choice in search engine use , 2009, SIGIR.

[53]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[54]  Scott Sanner,et al.  Social collaborative filtering for cold-start recommendations , 2014, RecSys '14.

[55]  Les MacLeod,et al.  Making SMART goals smarter. , 2012, Physician executive.

[56]  Alessandro Lazaric,et al.  Risk-Aversion in Multi-armed Bandits , 2012, NIPS.

[57]  Li Chen,et al.  A user-centric evaluation framework for recommender systems , 2011, RecSys '11.

[58]  Yong Tan,et al.  Feeling Blue? Go Online: An Empirical Study of Social Support among Patients , 2010, Inf. Syst. Res..

[59]  Paul E. Johnson,et al.  Understanding Variation in Chronic Disease Outcomes , 2002, Health care management science.

[60]  Qing Wang,et al.  Online Context-Aware Recommendation with Time Varying Multi-Armed Bandit , 2016, KDD.

[61]  Xiaoyan Zhu,et al.  Promoting Diversity in Recommendation by Entropy Regularizer , 2013, IJCAI.

[62]  Roger J. R. Levesque,et al.  Obesity and Overweight , 2011 .

[63]  Andrew E. B. Lim,et al.  Robust Multiarmed Bandit Problems , 2015, Manag. Sci..

[64]  Eric M. Schwartz,et al.  Dynamic Online Pricing with Incomplete Information Using Multi-Armed Bandit Experiments , 2018, Mark. Sci..

[65]  R. Snyderman,et al.  Improving health by taking it personally. , 2010, JAMA.