Using Cari Tools To Understand Measurement Error

Computer audio-recorded interviewing (CARI) has been used for more than a decade to detect field-interview fabrication, but it has the potential for much more. On the 2007 National Home and Hospice Care Survey, an innovative CARI system combined with behavior coding assessed problem- atic questions, monitored general compliance with protocols, and evaluated individual interviewer performance. This article discusses the benefits of CARI for recording large samples of interviews in a systematic, objective manner at reduced cost and less operational burden than traditional audio re- cording. The increased objectivity and volume of recordings allowed identi- fication of potential data-quality issues, specifically measurement error due to the questionnaire, the interviewer, and their interaction. CARI's potential as a tool for estimating the magnitude of the error in survey data is also discussed.

[1]  Seymour Sudman,et al.  Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research , 1995 .

[2]  Seymour Sudman,et al.  Measurement errors in surveys , 1993 .

[3]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[4]  F. S. P. Szuster,et al.  Nonsampling Error in Surveys , 1994 .

[5]  Charles F. Cannell,et al.  Using behavioral coding to identify cognitive problems with survey questions. , 1996 .

[6]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[7]  Paul P. Biemer,et al.  The Evolution of Audio Recording in Field Surveys , 2008 .

[8]  R. Groves,et al.  Survey Errors and Survey Costs. , 1991 .

[9]  I. Katz,et al.  Letting go of the rope--aggressive treatment, hospice care, and open access. , 2007, The New England journal of medicine.

[10]  Paul P. Biemer,et al.  Introduction to Survey Quality , 2003 .

[11]  J. Sim,et al.  The kappa statistic in reliability studies: use, interpretation, and sample size requirements. , 2005, Physical therapy.

[12]  R. Groves,et al.  Telephone Survey Methodology. , 1990 .

[13]  G. Willis,et al.  Cognitive Interviewing: A Tool for Improving Questionnaire Design , 2004 .

[14]  Laura Branden,et al.  Redesign and operation of the National Home And Hospice Care Survey, 2007. , 2010, Vital and health statistics. Ser. 1, Programs and collection procedures.

[15]  M. Rita Thissen,et al.  Using Computer Audio-Recorded Interviewing to Assess Interviewer Coding Error , 2008 .

[16]  Charles F. Cannell,et al.  A technique for evaluating interviewer performance : a manual for coding and analyzing interviewer behavior from tape recordings of household interviews , 1975 .