A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Significance Encouraging vaccination is a pressing policy problem. Our megastudy with 689,693 Walmart pharmacy customers demonstrates that text-based reminders can encourage pharmacy vaccination and establishes what kinds of messages work best. We tested 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination. Reminder texts increased vaccination rates by an average of 2.0 percentage points (6.8%) over a business-as-usual control condition. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts 3 d apart and stated that a vaccine was “waiting for you.” Forecasters failed to anticipate that this would be the best-performing treatment, underscoring the value of testing. Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was “waiting for you.” Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

[1]  C. Bulte,et al.  False Discovery in A/B Testing , 2021, Manag. Sci..

[2]  Katherine L. Milkman,et al.  Megastudies improve the impact of applied behavioural science , 2021, Nature.

[3]  Katherine L. Milkman,et al.  A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment , 2021, Proceedings of the National Academy of Sciences.

[4]  S. Saccardo,et al.  Behavioural nudges increase COVID-19 vaccinations , 2021, Nature.

[5]  Erez Yoeli,et al.  Digital Health Support in Treatment for Tuberculosis. , 2019, The New England journal of medicine.

[6]  Chad R. Mortensen,et al.  Trending Norms: A Lever for Encouraging Behaviors Performed by the Minority , 2019 .

[7]  Ron Berman,et al.  Test & Roll: Profit-Maximizing A/B Tests , 2018, Mark. Sci..

[8]  Ron Berman,et al.  Profit-Maximizing A/B Tests , 2018 .

[9]  N. Choudhry,et al.  Letters designed with behavioural science increase influenza vaccination in Medicare beneficiaries , 2018, Nature Human Behaviour.

[10]  Stefano DellaVigna,et al.  Predicting Experimental Results: Who Knows What? , 2016, Journal of Political Economy.

[11]  P. Effler,et al.  Randomized Controlled Trial of Text Message Reminders for Increasing Influenza Vaccination , 2017, The Annals of Family Medicine.

[12]  G. Walton,et al.  Dynamic Norms Promote Sustainable Behavior, Even if It Is Counternormative , 2017, Psychological science.

[13]  Katherine L. Milkman,et al.  Should Governments Invest More in Nudging? , 2017, Psychological science.

[14]  Charles F Manski,et al.  Sufficient trial size to inform clinical practice , 2016, Proceedings of the National Academy of Sciences.

[15]  David Huffman,et al.  Attention, intentions, and follow-through in preventive health behavior: Field experimental evidence on flu vaccination , 2015 .

[16]  Katherine L. Milkman,et al.  Using implementation intentions prompts to enhance influenza vaccination rates , 2011, Proceedings of the National Academy of Sciences.

[17]  G. Chapman,et al.  Opting in vs opting out of influenza vaccination. , 2010, JAMA.

[18]  T. Rex.  A practical guide to the American Community Survey (5-year estimates) , 2010 .

[19]  Jörg Stoye,et al.  Minimax regret treatment choice with finite samples , 2009 .

[20]  James J. Kellaris,et al.  The Influence of Humor Strength and Humor—Message Relatedness on Ad Memorability: A Dual Process Model , 2007 .

[21]  Cheng Li,et al.  Adjusting batch effects in microarray expression data using empirical Bayes methods. , 2007, Biostatistics.

[22]  Stacey R. Finkelstein,et al.  Recommendations Implicit in Policy Defaults , 2006, Psychological science.

[23]  R. Baer.  Conceptual and Empirical Review , 2006 .

[24]  G. Chapman,et al.  Moderators of the intention–behavior relationship in influenza vaccinations: Intention stability and unforeseen barriers , 2005 .

[25]  Praveen Aggarwal,et al.  Using Commitments to Drive Consistency: Enhancing the Effectiveness of Cause‐related Marketing Communications , 2005 .

[26]  P. Sheeran.  Intention—Behavior Relations: A Conceptual and Empirical Review , 2002 .

[27]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[28]  Xiao-Li Meng,et al.  Comparing correlated correlation coefficients , 1992 .

[29]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[30]  D. Kahneman,et al.  Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias , 1991 .

[31]  Daniel Kahneman,et al.  Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias , 1991 .

[32]  J. H. Steiger.  Tests for comparing elements of a correlation matrix. , 1980 .

[33]  O. J. Dunn,et al.  Correlation Coefficients Measured on the Same Individuals , 1969 .