The mystery shopper: a tool to measure public service delivery?

Originally the exclusive preserve of the private sector, the mystery shopper technique is increasingly being used in the public sector. In the wake of the reforms to modernise the state, accountability and performance-monitoring exercises are on the rise. They focus, in particular, on service quality and user-customer satisfaction. The article makes a twofold contribution to this topic: methodological and substantive. First of all, the article undertakes a scoping review of the literature on the mystery shopper. This review makes it possible to present the mystery shopper technique and its use in the public sector. For this bibliometric study, a sample of 34 papers was analysed. Second, the article offers a summary of the research into the mystery shopper technique, its potential and its limitations. Points for practitioners This article describes the use of the mystery shopper technique in the public sector. The areas for which mystery shopper surveys are commissioned are relatively limited, most being undertaken in the health sector. However, the scoping review emphasises the potential importance of mystery shopping for the purposes of the evaluation. As such, investigating other areas can be very interesting and promising for the public authorities. We also observe from this literature review that the challenges identified during mystery shopper studies can be overcome.

[1]  Peter Kuster,et al.  Handbook Of Practical Program Evaluation , 2016 .

[2]  Nathalie Schiffino,et al.  Risk Policies in the United States: Definition and Characteristics Based on a Scoping Review of the Literature , 2015, Risk analysis : an official publication of the Society for Risk Analysis.

[3]  Jan-Eric Furubo,et al.  The institutionalization of evaluation matters: Updating the International Atlas of Evaluation 10 years later , 2015 .

[4]  D. Roter,et al.  Impact of training on pharmacists' counseling of patients starting antidepressant therapy. , 2014, Patient education and counseling.

[5]  D. Collard Les enquêtes « client mystère » : une technique de contrôle des comportements , 2013 .

[6]  J. Maher,et al.  Breastfeeding guidance in community pharmacies: The results of a mystery shopper study , 2013 .

[7]  C. Bernsten,et al.  Non-prescription medicines for pain and fever--a comparison of recommendations and counseling from staff in pharmacy and general sales stores. , 2013, Health policy.

[8]  Mary Carter,et al.  Accessing primary care: a simulated patient study. , 2013, The British journal of general practice : the journal of the Royal College of General Practitioners.

[9]  J. Wolffsohn,et al.  The management of ocular allergy in community pharmacies in the United Kingdom , 2012, International Journal of Clinical Pharmacy.

[10]  K. Rhodes,et al.  Simulated patient studies: an ethical analysis. , 2012, The Milbank quarterly.

[11]  A. Bryant,et al.  Abortion misinformation from crisis pregnancy centers in North Carolina. , 2012, Contraception.

[12]  Vanda Marković-Peković,et al.  Self‐medication with antibiotics in the Republic of Srpska community pharmacies: pharmacy staff behavior , 2012, Pharmacoepidemiology and drug safety.

[13]  Jordy F. Gosselt,et al.  Improving shop floor compliance with age restrictions for alcohol sales: effectiveness of a feedback letter intervention. , 2012, European journal of public health.

[14]  D. Collard Les enquêtes client mystère: un dispositif de gestion au service des transformations organisationnelles ou comment réorganiser l'entreprise " au nom du client " , 2012 .

[15]  Belinda Battley,et al.  Finding and Addressing the Gaps: Two Evaluations of Archival Reference Services , 2012 .

[16]  A. Dembe,et al.  The Use of a “Mystery Shopper” Methodology to Evaluate Children’s Access to Psychiatric Services , 2012, The Journal of Behavioral Health Services & Research.

[17]  Improving Outpatient Access and Patient Experiences in Academic Ambulatory Care , 2012, Academic medicine : journal of the Association of American Medical Colleges.

[18]  Jordy F. Gosselt,et al.  Why should i comply? Sellers' accounts for (non-)compliance with legal age limits for alcohol sales , 2012, Substance Abuse Treatment, Prevention, and Policy.

[19]  D. Severt,et al.  Initial and Ongoing Mystery Shopper Motivations: An Empirical Investigation By Experience Level and an Application of the Theory of Motivational Crowding , 2012 .

[20]  K. Rhodes Taking the mystery out of "mystery shopper" studies. , 2011, The New England journal of medicine.

[21]  G. Latham,et al.  Mystery shoppers: A new tool for coaching employee performance improvement , 2011 .

[22]  Jordy F. Gosselt,et al.  Effects of a national information campaign on compliance with age restrictions for alcohol sales. , 2011, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[23]  R. Pollini,et al.  Not sold here: limited access to legally available syringes at pharmacies in Tijuana, Mexico , 2011, Harm reduction journal.

[24]  W. DeJong,et al.  The Effect of Mystery Shopper Reports on Age Verification for Tobacco Purchases , 2011, Journal of health communication.

[25]  M. Jankalová,et al.  Mystery Shopping − The Tool of Employee Communication Skills Evaluation , 2011 .

[26]  Damien Collard De la stratégie orientée client aux « enquêtes client mystère » , 2010 .

[27]  A. Glasier,et al.  Community pharmacists providing emergency contraception give little advice about future contraceptive use: a mystery shopper study. , 2010, Contraception.

[28]  D. Levac,et al.  Scoping studies: advancing the methodology , 2010, Implementation science : IS.

[29]  Duncan Dickson,et al.  A Conceptual Model for Mystery Shopping Motivations , 2010 .

[30]  Claudio Straccia,et al.  Les “achats tests”. Etude des comportements de vente d'alcool aux mineurs en Suisse (2000-2008). , 2010 .

[31]  M. Weiss,et al.  Use of simulated patients to assess the clinical and communication skills of community pharmacists , 2010, Pharmacy World & Science.

[32]  S. Simoens Analysis of Drug Authentication at the Point of Dispensing in Belgian and Greek Community Pharmacies , 2009, The Annals of pharmacotherapy.

[33]  Améliorer la qualité grâce aux prix de la qualité organisés par voie de concours ? Un cadre d'évaluation de l'impact , 2009 .

[34]  C. Schneider,et al.  Measuring the Assessment and Counseling Provided with the Supply of Nonprescription Asthma Reliever Medication: A Simulated Patient Study , 2009, The Annals of pharmacotherapy.

[35]  Steve Jacob Opération chloroforme ou la réinvention de l’État rationnel : l’évaluation et les données probantes , 2009 .

[36]  Ronan Divard Les enquêtes mystère : une technique délicate à mettre en oeuvre , 2009 .

[37]  É. Charbonneau,et al.  La situation de l'amélioration de la qualité des services publics : Vingt-cinq années de tendances et de pratiques aux États-Unis , 2009 .

[38]  M. Moll,et al.  Selling alcohol to underage adolescents in Romania: compliance with age restrictions in Pitesti , 2009 .

[39]  Stephen J. Romano,et al.  Role Playing , 2008, Behavior modification.

[40]  A. Douglas,et al.  The Impact of Mystery Customers on Employees , 2008 .

[41]  Jordy F. Gosselt,et al.  Mystery shopping and alcohol sales: do supermarkets and liquor stores sell alcohol to underage customers? , 2007, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[42]  D. Alte,et al.  Evaluation of Consultation in Community Pharmacies with Mystery Shoppers , 2007, The Annals of pharmacotherapy.

[43]  Edwin Van Troostenberghe Wie is bang voor ROB? De open bare bibliotheek doorgelicht , 2007 .

[44]  Enrichir l'approche théorique de la qualité dans les services: qualité du service et qualité de service , 2006 .

[45]  S. Sykes,et al.  A ‘mystery shopper’ project to evaluate sexual health and contraceptive services for young people in Croydon , 2006, Journal of Family Planning and Reproductive Health Care.

[46]  M. Chauviére Que reste-t-il de la ligne jaune entre l'usager et le client ? , 2006 .

[47]  C. M. Saunders Effectiveness of the Toll-Free Line for Public Insurance Programs , 2005, Evaluation & the health professions.

[48]  P. Calvert It's a mystery , 2005 .

[49]  Sandra Mathison,et al.  Encyclopedia of Evaluation , 2004 .

[50]  P. Norris Purchasing restricted medicines in New Zealand pharmacies: results from a "mystery shopper" study , 2002, Pharmacy World and Science.

[51]  Steve Jacob,et al.  Cheminement institutionnel de l'évaluation des politiques publiques en France, en Suisse et aux Pays-Bas (1970-2003) , 2004 .

[52]  Sophie Botte La qualité de service public : une typologie des « mondes de la qualité » de La Poste , 2004 .

[53]  J. Jesson Mystery shopping demystified: is it a justifiable research method? , 2004 .

[54]  Wendy M. Bennett,et al.  Pharmacists' knowledge and the difficulty of obtaining emergency contraception. , 2003, Contraception.

[55]  M. Bradbury,et al.  Measuring Customer Service: Georgia's Local Government Mystery Shopper Program , 2003 .

[56]  M. Amiel La qualité de services dans les administrations publiques : un défi du changement , 2003 .

[57]  Mathilda Ruwe,et al.  Preventing Illegal Tobacco and Alcohol Sales to Minors through Electronic Age-Verification Devices: A Field Effectiveness Study , 2003, Journal of public health policy.

[58]  John D. Donahue,et al.  Market-Based Governance: Supply Side, Demand Side, Upside, and Downside , 2002 .

[59]  S. Osborne The foundations of contemporary public management , 2002 .

[60]  Alan Wilson,et al.  Mystery shopping: Using deception to measure service performance , 2001 .

[61]  Alan I. Wilson The use of mystery shopping in the measurement of service delivery , 1998 .

[62]  A. Colman,et al.  Mystery Customer Research: Cognitive Processes Affecting Accuracy , 1997 .

[63]  A. Pettigrew,et al.  Characterizing the ‘New Public Management’ , 1996 .

[64]  Philip Kotler Pour une réussite du marketing des services publics dans les années 90 , 1993 .