The Choice Architecture of School Choice Websites

Abstract We conducted a randomized factorial experiment to determine how displaying school information to parents in different ways affects what schools they choose for their children in a hypothetical school district. In a sample of 3,500 low-income parents of school-aged children, a small design manipulation, such as changing the default order in which schools were presented, induced meaningful changes in the types of schools selected. Other design choices such as using icons to represent data, instead of graphs or just numbers, or presenting concise summaries instead of detailed displays, also led parents to choose schools with higher academic performance. We also examined effects on parents’ understanding of the information and their self-reported satisfaction and ease of use. In some cases, there were trade-offs. For example, representing data using only numbers maximized understanding, but adding graphs maximized satisfaction at the expense of understanding.

[1]  Maya Bar-Hillel,et al.  Position Effects in Choice From Simultaneous Displays , 2015, Perspectives on psychological science : a journal of the Association for Psychological Science.

[2]  Paul Slovic,et al.  Strategies for reporting health plan performance information to consumers: evidence from controlled studies. , 2002, Health services research.

[3]  Andrew Saultz,et al.  Informing or Shaping Public Opinion? The Influence of School Accountability Data Format on Public Perceptions of School Quality , 2014, American Journal of Education.

[4]  Miles Richardson,et al.  An objective examination of consumer perception of nutrition information based on healthiness ratings and eye movements , 2007, Public Health Nutrition.

[5]  Gordon D. A. Brown,et al.  Decision by sampling , 2006, Cognitive Psychology.

[6]  Stepan Jurajda,et al.  Admission to Selective Schools, Alphabetically , 2005 .

[7]  Jinhong Xie,et al.  Third-Party Product Review and Firm Marketing Strategy , 2005 .

[8]  Ellen Goldring,et al.  Choosing Indianapolis Charter Schools: Espoused versus Revealed Academic Preferences. Research Brief. , 2010 .

[9]  Robert Sugden,et al.  On Nudging: A Review of Nudge: Improving Decisions About Health, Wealth and Happiness by Richard H. Thaler and Cass R. Sunstein , 2009 .

[10]  Mark Schneider,et al.  Choosing Schools: Consumer Choice and the Quality of American Schools , 2000 .

[11]  Norbert Schwarz,et al.  Constructing reality and its alternatives: an inclusion/ exclusion model of assimilation and contrast effects in social judgment , 1992 .

[12]  Justine S. Hastings,et al.  Heterogeneous Preferences and the Efficacy of Public School Choice , 2008 .

[13]  Jon Valant,et al.  My Kids, Your Kids, Our Kids: What Parents and the Public Want from Schools , 2017, Teachers College Record: The Voice of Scholarship in Education.

[14]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[15]  G Chakraborty,et al.  How consumers choose health insurance. , 1994, Journal of health care marketing.

[16]  A. Parducci.  Category judgment: a range-frequency model. , 1965, Psychological review.

[17]  Justine S. Hastings,et al.  Information, School Choice, and Academic Achievement: Evidence from Two Experiments , 2007 .

[18]  Mark Schneider,et al.  Shopping for Schools: In the Land of the Blind, The One-Eyed Parent May be Enough , 1998 .

[19]  Chris Arney.  Nudge: Improving Decisions about Health, Wealth, and Happiness , 2015 .

[20]  Joachim Westenhoefer,et al.  Impact of different food label formats on healthiness evaluation and food choice of consumers: a randomized-controlled study , 2009, BMC public health.

[21]  Nancy L. Kocovski,et al.  Perception of Quantitative Information for Treatment Decisions , 2000, Medical decision making : an international journal of the Society for Medical Decision Making.

[22]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[23]  A. Gelman.  Analysis of variance: Why it is more important than ever? , 2005, math/0504499.

[24]  Jon Valant,et al.  The Word on the Street or the Number from the State? Government-Provided Information and Americans’ Opinions of Schools , 2020 .

[25]  K. Muralidharan,et al.  Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments , 2019, SSRN Electronic Journal.

[26]  Patrick Paul Walsh,et al.  Does Choice Increase Information? Evidence from Online School Search Behavior , 2017 .

[27]  Jakob Nielsen,et al.  Prioritizing Web Usability , 2006 .

[28]  Justine S. Hastings,et al.  Preferences and Heterogeneous Treatment Effects in a Public School Choice Lottery , 2006 .

[29]  Richard H. Thaler,et al.  Design Choices in Privatized Social-Security Systems: Learning from the Swedish Experience , 2004 .

[30]  Sean P. Corcoran,et al.  The School Choice Journey: School Vouchers and the Empowerment of Urban Families by ThomasStewart and Patrick J.Wolf. New York, Palgrave, 2014. 236 pp. $100.00. , 2016 .

[31]  A. Tversky,et al.  Loss Aversion in Riskless Choice: A Reference-Dependent Model , 1991 .

[32]  J. Krosnick,et al.  The Impact of Candidate Name Order on Election Outcomes , 1998 .

[33]  Mario Pandelaere,et al.  When and why attribute sorting affects attribute weights in decision-making , 2014 .

[34]  J. Gabry,et al.  Bayesian Applied Regression Modeling via Stan , 2016 .

[35]  Helena Szrek,et al.  Choice Set Size and Decision Making: The Case of Medicare Part D Prescription Drug Plans , 2010, Medical decision making : an international journal of the Society for Medical Decision Making.

[36]  Eric J. Johnson,et al.  The Construction of Preference: Do Defaults Save Lives? , 2006 .

[37]  Iris Vessey,et al.  Cognitive Fit: A Theory‐Based Analysis of the Graphs Versus Tables Literature* , 1991 .

[38]  Ira Nichols-Barrer,et al.  Beyond “Treatment Versus Control”: How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research , 2019, Evaluation review.

[39]  S. Waslander,et al.  MARKETS IN EDUCATION: AN ANALYTICAL REVIEW OF EMPIRICAL RESEARCH ON MARKET MECHANISMS IN EDUCATION , 2010 .

[40]  Jack Schneider,et al.  Beyond Test Scores: A Better Way to Measure School Quality , 2017 .

[41]  P. Gustafson,et al.  Conservative prior distributions for variance parameters in hierarchical models , 2006 .

[42]  Eric J. Johnson,et al.  Medicine. Do defaults save lives? , 2003, Science.

[43]  Sarah R. Cohodes,et al.  Leveling the Playing Field for High School Choice: Results from a Field Experiment of Informational Interventions , 2018 .

[44]  Courtney A. Bell.  All Choices Created Equal? The Role of Choice Sets in the Selection of Schools , 2009 .

[45]  Gerardine DeSanctis,et al.  COMPUTER GRAPHICS AS DECISION AIDS: DIRECTIONS FOR RESEARCH* , 1984 .

[46]  C. K. Mertz,et al.  Less Is More in Presenting Quality Information to Consumers , 2007, Medical care research and review : MCRR.

[47]  B. Gross,et al.  Common Enrollment, Parents, and School Choice: Early Evidence from Denver and New Orleans. Making School Choice Work Series. , 2015 .

[48]  David Hammond,et al.  Nutrition labels on pre-packaged foods: a systematic review , 2011, Public Health Nutrition.

[49]  Lynn Bosetti,et al.  Determinants of school choice: understanding how parents choose elementary schools in Alberta , 2004 .

[50]  Patrick J. Wolf,et al.  The School Choice Journey , 2014 .

[51]  Mette Trier Damgaard,et al.  Nudging in Education , 2018, Economics of Education Review.

[52]  R. A. Waller,et al.  A Bayes Rule for the Symmetric Multiple Comparisons Problem , 1969 .

[53]  Mark Schneider,et al.  What Research Can Tell Policymakers about School Choice , 2001 .

[54]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[55]  Alan Collins,et al.  Parental preferences and choice of school , 2000 .

[56]  Sarah A. Nagro,et al.  The Readability and Complexity of District-Provided School-Choice Information , 2015 .

[57]  Ginger Zhe Jin,et al.  The Case in Support of Restaurant Hygiene Grade Cards , 2005 .

[58]  P. Shekelle,et al.  Web appendix A , 2013 .

[59]  Andrew Gelman,et al.  Why We (Usually) Don't Have to Worry About Multiple Comparisons , 2009, 0907.2478.

[60]  S. Glazerman,et al.  Market Signals: Evidence on the Determinants and Consequences of School Choice From a Citywide Lottery , 2016 .

[61]  R. Neild,et al.  Parent Management of School Choice in a Large Urban District , 2005 .