Evidence of non-response bias in the Press-Ganey patient satisfaction survey

BackgroundMeasures of patient satisfaction have gained prominence in recent years as changes to the American health care system have led to the incorporation of such metrics into reimbursement models. The response rate for widely-used outpatient satisfaction metrics and variables influencing the probability of survey nonresponse remain largely unknown.MethodsWe reviewed all unique adult patients (16,779) who completed an outpatient encounter in the Department of Orthopaedic surgery at our academic institution from 1/1/13 to 10/24/13. Survey data was linked to each clinic visit, and patient factors including age, sex, insurance type, zip code, and orthopaedic subspecialty visited were recorded.The overall survey response rate was calculated. Logistic regression was performed, and unadjusted and adjusted odds ratios of patients’ probability of responding to the Press-Ganey survey were calculated.ResultsTwo thousand seven hundred sixty two (16.5 %) of individuals completed a Press-Ganey patient satisfaction survey and 14017 patients did not respond. For those patients considered responders, 906 patients (32.8 %) did not complete all the survey items. Among these 906 patients, the mean number of missing items was 2.24 (Standard Deviation SD: 2.19).Age, sex, insurance type, and orthopaedic subspecialty were all found to be associated with the odds of responding to our patient satisfaction survey. Advancing age increased the odds of responding to the survey (Adjusted Odds Ratio (OR) = 3.396 for ≥65 vs. 18–29, p < 0.001). Several variables were associated with a decreased odds of survey response, and included male sex (Adjusted OR = 0.782 for Males vs. Females, p < 0.001), insurance type (Adjusted OR = 0.311 for Medicaid/Self-Pay vs. Private), and subspecialty type (Adjusted OR = 0.623 for Trauma vs. Adult Reconstruction).ConclusionsThe response rate to the Press-Ganey Medical Practice Survey of outpatient satisfaction is low in an orthopaedic outpatient population, and furthermore, is impacted by patient characteristics such as age, sex, insurance type, and type of orthopaedic subspecialist encountered. The findings of the present study should inform future non-response weighting procedures in this area. More research is needed to assess non-response bias—including follow-up studies of non-respondents—in order to more accurately measure of patient satisfaction.

[1]  G. Young,et al.  Patient satisfaction with hospital care: effects of demographic and institutional characteristics. , 2000, Medical care.

[2]  Peter Bower,et al.  Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England , 2009, BMJ : British Medical Journal.

[3]  C. Howie,et al.  What determines patient satisfaction with surgery? A prospective cohort study of 4709 patients following total joint replacement , 2013, BMJ Open.

[4]  Robert M. Groves,et al.  The Impact of Nonresponse Rates on Nonresponse Bias A Meta-Analysis , 2008 .

[5]  M. Swiontkowski,et al.  Measuring patient satisfaction in orthopaedic surgery. , 2015, The Journal of bone and joint surgery. American volume.

[6]  T V Perneger,et al.  Analysis of non-response bias in a mailed health survey. , 1997, Journal of clinical epidemiology.

[7]  Jane M. Young,et al.  Predictors of patient and surgeon satisfaction after orthopaedic trauma. , 2009, Injury.

[8]  Cathal Doyle,et al.  A systematic review of evidence on the links between patient experience and clinical safety and effectiveness , 2013, BMJ Open.

[9]  Tse-Hua Shih,et al.  Comparing Response Rates from Web and Mail Surveys: A Meta-Analysis , 2008 .

[10]  A. Vranceanu,et al.  Factors associated with patient satisfaction. , 2011, The Journal of hand surgery.

[11]  A. Zaslavsky,et al.  Examining the Role of Patient Experience Surveys in Measuring Health Care Quality , 2014, Medical care research and review : MCRR.

[12]  G. Simon,et al.  Are comparisons of consumer satisfaction with providers biased by nonresponse or case-mix differences? , 2009, Psychiatric services.

[13]  P. Cleary,et al.  The effect of response scale, administration mode, and format on responses to the CAHPS Clinician and Group survey. , 2014, Health services research.

[14]  A. Zaslavsky,et al.  Adjusting for Patient Characteristics When Analyzing Reports From Patients About Hospital Care , 2001, Medical care.

[15]  M. Davern,et al.  Nonresponse rates are a problematic indicator of nonresponse bias in survey research. , 2013, Health services research.

[16]  T. Osler,et al.  Factors associated with patient satisfaction scores for physician care in trauma patients , 2013, The journal of trauma and acute care surgery.

[17]  A. Jha,et al.  Patients' perception of hospital care in the United States. , 2008, The New England journal of medicine.

[18]  Matthew P. Manary,et al.  The patient experience and health outcomes. , 2013, The New England journal of medicine.

[19]  R. Jones,et al.  Health surveys in the workplace: comparison of postal, email and World Wide Web methods. , 1999, Occupational medicine.

[20]  Anthony F Jerant,et al.  The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. , 2012, Archives of internal medicine.

[21]  B. Snaith,et al.  Estimating the Effect of Nonresponse Bias in a Survey of Hospital Organizations , 2013, Evaluation & the health professions.

[22]  Nick Black,et al.  Relationship between patient reported experience (PREMs) and patient reported outcomes (PROMs) in elective surgery , 2014, BMJ quality & safety.

[23]  Matthew P. Manary,et al.  Patient Satisfaction and Its Relationship With Clinical Quality and Inpatient Mortality in Acute Myocardial Infarction , 2010, Circulation. Cardiovascular quality and outcomes.

[24]  Timothy P. Johnson,et al.  Response rates and nonresponse errors in surveys. , 2012, JAMA.

[25]  S. Ishman,et al.  Whose experience is measured?: A pilot study of patient satisfaction demographics in pediatric otolaryngology , 2014, The Laryngoscope.

[26]  Robert D. Tortora,et al.  Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet , 2009 .

[27]  A. Zaslavsky,et al.  Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. , 2009, Health services research.

[28]  A. Zaslavsky,et al.  Factors Affecting Response Rates to the Consumer Assessment of Health Plans Study Survey , 2002, Medical care.

[29]  M. Elliott,et al.  Patterns of unit and item nonresponse in the CAHPS Hospital Survey. , 2005, Health services research.

[30]  A. Zaslavsky,et al.  Should Health Care Providers be Accountable for Patients’ Care Experiences? , 2015, Journal of General Internal Medicine.

[31]  M. McGirt,et al.  Determining the quality and effectiveness of surgical spine care: patient satisfaction is not a valid proxy. , 2013, The spine journal : official journal of the North American Spine Society.

[32]  C. Saltzman,et al.  Association Between Orthopaedic Outpatient Satisfaction and Non-Modifiable Patient Factors. , 2015, The Journal of bone and joint surgery. American volume.

[33]  Kennon R. Copeland,et al.  Quantifying bias in a health survey: Modeling total survey error in the National Immunization Survey , 2011, Statistics in medicine.