Evaluating survey quality in health services research: a decision framework for assessing nonresponse bias.

OBJECTIVE To address the issue of nonresponse as problematic and offer appropriate strategies for assessing nonresponse bias. STUDY DESIGN A review of current strategies used to assess the quality of survey data and the challenges associated with these strategies is provided along with appropriate post-data collection techniques that researchers should consider. PRINCIPAL FINDINGS Response rates are an incomplete assessment of survey data quality, and quick reactions to response rate should be avoided. Based on a five-question decision making framework, we offer potential ways to assess nonresponse bias, along with a description of the advantages and disadvantages to each. CONCLUSIONS It is important that the quality of survey data be considered to assess the relative contribution to the literature of a given study. Authors and funding agencies should consider the potential effects of nonresponse bias both before and after survey administration and report the results of assessments of nonresponse bias in addition to response rates.

[1]  D. Asch,et al.  Lottery-based versus fixed incentives to increase clinicians' response to surveys. , 2011, Health services research.

[2]  Keith Nicholls,et al.  Enhancing response rates in physician surveys: the limited utility of electronic options. , 2011, Health services research.

[3]  T. Koepsell,et al.  Comparison of telephone with World Wide Web-based responses by parents and teens to a follow-up survey after injury. , 2011, Health services research.

[4]  L. Haas,et al.  Health Insurance Portability and Accountability Act (HIPAA) Authorization and Survey Nonresponse Bias , 2011, Medical care.

[5]  T. Beebe,et al.  Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys. , 2010, Health services research.

[6]  Thomas G. McLeod,et al.  Personality characteristics of health care satisfaction survey non-respondents. , 2009, International journal of health care quality assurance.

[7]  T. Beebe,et al.  Survey-related experiential and attitudinal correlates of future health survey participation: results of a statewide survey. , 2008, Mayo Clinic proceedings.

[8]  Robert M. Groves,et al.  The Impact of Nonresponse Rates on Nonresponse Bias A Meta-Analysis , 2008 .

[9]  M. Partin,et al.  Mixed-mode administration reduced bias and enhanced poststratification adjustments in a health behavior survey. , 2007, Journal of clinical epidemiology.

[10]  T. Taris,et al.  How may nonresponse affect findings in organizational surveys? The tendency-to-the-positive effect , 2007 .

[11]  T. Beebe,et al.  Mixing web and mail methods in a survey of physicians. , 2007, Health services research.

[12]  S. Rogelberg,et al.  Introduction Understanding and Dealing With Organizational Survey Nonresponse , 2007 .

[13]  J. Yessis,et al.  Initial versus Prompted Responders to Patient Satisfaction Surveys: Implication for Interpretation of Patient Feedback , 2006 .

[14]  R. Groves,et al.  The Role of Topic Interest in Survey Participation Decisions , 2004 .

[15]  William E. Knight,et al.  Profiling active and passive nonrespondents to an organizational survey. , 2003, The Journal of applied psychology.

[16]  Chen-Chung Ma,et al.  Factors That Influence Nurses’ Job Satisfaction , 2003, The Journal of nursing administration.

[17]  J. Gurwitz,et al.  A demonstration of the impact of response bias on the results of patient satisfaction surveys. , 2002, Health services research.

[18]  D. Asch,et al.  Randomized trial of 5 dollars versus 10 dollars monetary incentives, envelope size, and candy to increase physician response rates to mailed questionnaires. , 2002, Medical care.

[19]  D. Asch,et al.  Randomized Trial of $5 Versus $10 Monetary Incentives, Envelope Size, and Candy to Increase Physician Response Rates to Mailed Questionnaires , 2002 .

[20]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data: Little/Statistical Analysis with Missing Data , 2002 .

[21]  Alan M. Zaslavsky,et al.  Using Telephone Interviews to Reduce Nonresponse Bias to Mail Surveys of Health Plan Members , 2002, Medical care.

[22]  T. Konrad,et al.  Reported response rates to mailed physician questionnaires. , 2001, Health services research.

[23]  Milton D. Hakel,et al.  Attitudes toward Surveys: Development of a Measure and Its Relationship to Respondent Behavior , 2001 .

[24]  E. Singer,et al.  Leverage-saliency theory of survey participation: description and an illustration. , 2000, Public opinion quarterly.

[25]  N Wood,et al.  Response rate in patient satisfaction research: an analysis of 210 published studies. , 1998, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[26]  E. Kaner,et al.  'So much post, so busy with practice--so, no time!': a telephone survey of general practitioners' reasons for not participating in postal questionnaire surveys. , 1998, The British journal of general practice : the journal of the Royal College of General Practitioners.

[27]  L A Lillard,et al.  Linking Medicare and National Survey Data , 1997, Annals of Internal Medicine.

[28]  D A Asch,et al.  Response rates to mail surveys published in medical journals. , 1997, Journal of clinical epidemiology.

[29]  Y. Baruch,et al.  Back to square zero: The post-corporate career , 1997 .

[30]  A. Deehan,et al.  The effect of cash and other financial inducements on the response rate of general practitioners in a national postal study. , 1997, The British journal of general practice : the journal of the Royal College of General Practitioners.

[31]  P. Roth.  MISSING DATA: A CONCEPTUAL REVIEW FOR APPLIED PSYCHOLOGISTS , 1994 .

[32]  Robert M. Groves,et al.  UNDERSTANDING THE DECISION TO PARTICIPATE IN A SURVEY , 1992 .

[33]  Ruth C. Engs,et al.  Responders vs. Non-Responders to a Mail Survey: Are They Different? , 1992, Canadian Journal of Program Evaluation.

[34]  Subir Ghosh,et al.  Statistical Analysis With Missing Data , 1988 .

[35]  L. Fuller,et al.  I Hear You Knocking But You Can't Come In , 1982 .

[36]  Terry S. Overton,et al.  Estimating Nonresponse Bias in Mail Surveys , 1977 .

[37]  F. Filion,et al.  Exploring and Correcting for Nonresponse Bias Using Follow-ups of Non Respondents , 1976 .

[38]  R. A. Ellis,et al.  The Use of Potential Nonrespondents for Studying Nonresponse Bias , 1970 .

[39]  R. Groves.  Nonresponse Rates and Nonresponse Bias in Household Surveys , 2006 .

[40]  A. Luong,et al.  Employee attitude surveys: examining the attitudes of noncompliant employees. , 2000, The Journal of applied psychology.

[41]  N. C. Schaeffer,et al.  Institute for Research on Poverty Discussion Paper no. 1024-93 Using Survey Participants to Estimate the Impact of Nonparticipation , 1993 .

[42]  Carol P. Sosdian,et al.  Nonresponse in Mail Surveys: Access Failure or Respondent Resistance , 1980 .