Calendar and Question-List Survey Methods: Association Between Interviewer Behaviors and Data Quality

Insights are provided on the role of retrieval and conversational properties of Event History Calendars (EHCs) that promote higher quality retrospective reports than do standardized question-list (Q-list) methods. A verbal behavior coding analysis of 218 EHC and 197 Q-list interviews revealed significantly more behaviors in the EHC condition that indicated the use of timeline retrieval strategies and conversational engagement. Analyses of data quality measures demonstrate that there is not a significantly greater degree of interviewer variation on data quality in the EHC method. Better data quality was associated with a higher prevalence of retrieval cues, a greater degree of response openness, and lower levels of cognitive difficulty and rapport. The association of data quality and verbal behavior also interacted with method: retrieval cues and cognitive difficulty were directly associated with EHC response quality and indirectly associated with Q-list quality; rapport behaviors had a more detrimental effect on Q-list data quality.

[1]  E. Loftus,et al.  Retrieving Multiple Autobiographical Memories , 1985 .

[2]  Hanneke Houtkoop-Steenstra.  Interaction and the Standardized Survey Interview: The standardized survey interview , 2000 .

[3]  F. Conrad,et al.  Clarifying question meaning in a household telephone survey. , 2000, Public opinion quarterly.

[4]  Martin A. Conway,et al.  Remembering our past: Autobiographical knowledge and autobiographical memories , 1996 .

[5]  Wil Dijkstra,et al.  Testing Questionnaires Using Interaction Coding , 2002 .

[6]  R. Belli,et al.  The structure of autobiographical memory and the event history calendar: potential improvements in the quality of retrospective reports in surveys. , 1998, Memory.

[7]  Leroy Bailey,et al.  Measures of Interviewer Bias and Variance , 1977 .

[8]  Leslie Kish,et al.  Studies of Interviewer Variance for Attitudinal Variables , 1962 .

[9]  F. J. Fowler,et al.  How unclear terms affect survey data. , 1992, Public opinion quarterly.

[10]  Jennifer Dykema,et al.  The Effect of Interviewer and Respondent Behavior on Data Quality: Analysis of Interaction Coding in a Validation Study , 1997 .

[11]  R. Fivush,et al.  Autobiographical knowledge and autobiographical memories , 1996 .

[12]  Frederick G. Conrad,et al.  Does Conversational Interviewing Reduce Survey Measurement Error , 1997 .

[13]  Charles F. Cannell,et al.  A Technique for Evaluating Interviewer Performance. , 1976 .

[14]  L. Barsalou.  The content and organization of autobiographical memories , 1988 .

[15]  Charles F. Cannell,et al.  Using behavioral coding to identify cognitive problems with survey questions. , 1996 .

[16]  Stanley Presser,et al.  SURVEY PRETESTING: DO DIFFERENT METHODS PRODUCE DIFFERENT RESULTS? , 1994 .

[17]  F P Stafford,et al.  Event history calendars and question list surveys: a direct comparison of interviewing methods. , 2001, Public opinion quarterly.

[18]  Wil Dijkstra.  Interviewing Style and Respondent Behavior , 1987 .

[19]  E F Loftus,et al.  Recall strategies and memory for health-care visits. , 1990, The Milbank quarterly.

[20]  J. Williams,et al.  INTERVIEWER ROLE PERFORMANCE: A FURTHER NOTE ON BIAS IN THE INFORMATION INTERVIEW , 1968 .

[21]  R. Groves,et al.  Survey Errors and Survey Costs. , 1991 .

[22]  P. Beatty,et al.  Understanding the standardized/non-standardized interviewing controversy. , 1999 .

[23]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[24]  Wil Dijkstra,et al.  Methods of behavior coding of survey interviews , 2006 .

[25]  G. Willis,et al.  The use of verbal report methods in the development and testing of survey questionnaires , 1991 .