The Nature of Nonresponse in a Medicaid Survey: Causes and Consequences

This study examines the effect of multiple phases of data collection on 1) the representativeness of the respondents and 2) survey responses to substantive items. A version of the CAHPS survey instrument was used to collect information from 1,600 adults covered by Massachusetts Medicaid about their experiences with getting health care through their health plans. In order to gain a better understanding of the significance of nonresponse, intensive follow-up with nonrespondents was done utilizing three phases of data collection: mail, telephone, and inperson protocols in successive waves of contact. Using administrative data, we were able to compare demographic and health characteristics of respondents to each phase, as well as cumulatively, with the characteristics of the total sample. The results indicate that the addition of completed surveys from each successive phase of data collection yielded a final group of respondents that is more representative of the total population with respect to age, race, presence of a chronic condition, and annual medical costs. Responses to 40 key survey items were also analyzed by phase and cumulatively. Responses from mail and telephone respondents were significantly different for eleven of these items; responses collected from in-person interviews were significantly different from the combined data from mail and telephone respondents for three of these items.

[1]  P. Chisnall.  Mail and Internet Surveys: The Tailored Design Method , 2007, Journal of Advertising Research.

[2]  F. J. Fowler,et al.  Comparing telephone and mail responses to the CAHPS survey instrument. Consumer Assessment of Health Plans Study. , 1999, Medical care.

[3]  Alan M. Zaslavsky,et al.  Using Telephone Interviews to Reduce Nonresponse Bias to Mail Surveys of Health Plan Members , 2002, Medical care.

[4]  G. Rosenthal,et al.  An evaluation of the impact of nonresponse bias on patient satisfaction surveys. , 1997, Medical care.

[5]  J. R. Hochstim.  A Critical Comparison of Three Strategies of Collecting Data from Households , 1967 .

[6]  C. McHorney,et al.  Comparisons of the Costs and Quality of Norms for the SF-36 Health Survey Collected by Mail Versus Telephone Interview: Results From a National Survey , 1994, Medical care.

[7]  J Siemiatycki,et al.  Nonresponse bias and early versus all responders in mail and telephone surveys. , 1984, American journal of epidemiology.

[8]  Edith D. de Leeuw,et al.  A comparison of nonresponse in mail, telephone, and face-to-face surveys , 1994 .

[9]  Susan Sprachman,et al.  Costs and Benefits of Improving Response Rates for a Hard-to-Reach Population , 2003 .

[10]  R. Groves,et al.  Survey Errors and Survey Costs. , 1991 .

[11]  R M Groves,et al.  Consequences of reducing nonresponse in a national telephone survey. , 2000, Public opinion quarterly.

[12]  Floyd Jackson Fowler.  NONRESPONSE BIAS IN MAIL SURVEYS OF HEALTH PLAN MEMBERS , 2002 .

[13]  E. Singer,et al.  Leverage-saliency theory of survey participation: description and an illustration. , 2000, Public opinion quarterly.

[14]  E. Singer,et al.  The effects of response rate changes on the index of consumer sentiment. , 2000, Public opinion quarterly.