Using Cognitive Interviewing and Behavioral Coding to Determine Measurement Equivalence across Linguistic and Cultural Groups

This study examined and compared results from two questionnaire pretesting methods (i.e., behavioral coding and cognitive interviewing [CI]) to assess systematic measurement bias in survey questions for adult smokers across six countries (United States, Australia, Uruguay, Mexico, Malaysia, and Thailand). Protocol development and translation involved multiple bilingual partners in each linguistic/cultural group. The study was conducted with convenience samples of 20 adult smokers in each country. Behavioral coding and CI methods produced similar conclusions regarding measurement bias for some questions; however, CI was more likely to identify potential response errors than behavioral coding. Coordinated qualitative pretesting of survey questions (or postsurvey evaluation) is feasible across cultural groups and can provide important information on comprehension and comparability. The CI appears to be a more robust technique than behavioral coding, although combinations of the two might be even better.

[1]  K. Seers.  Qualitative data analysis , 2011, Evidence Based Nursing.

[2]  J. Madans,et al.  Results of a cross-national structured cognitive interviewing protocol to test measures of disability , 2011 .

[3]  James F. Thrasher,et al.  Smoke-free policies and the social acceptability of smoking in Uruguay and Mexico: findings from the International Tobacco Control Policy Evaluation Project. , 2009, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[4]  D. Feaster,et al.  Measurement and data analysis in research addressing health disparities in substance abuse. , 2009, Journal of substance abuse treatment.

[5]  Deirdre Lawrence,et al.  Translation of a tobacco survey into Spanish and Asian languages: the Tobacco Use Supplement to the Current Population Survey. , 2008, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[6]  E. Zahnd,et al.  Questionnaire Design from a Cross-Cultural Perspective: An Empirical Investigation of Koreans and Non-Koreans , 2007, Journal of health care for the poor and underserved.

[7]  T. Lam.  IARC Handbooks of cancer prevention: Tobacco control, v. 11: Reversal of risk after quitting smoking , 2007 .

[8]  S. Gregorich.  Do Self-Report Instruments Allow Meaningful Comparisons Across Diverse Population Groups?: Testing Measurement Invariance Using the Confirmatory Factor Analysis Framework , 2006, Medical care.

[9]  Young Ik Cho,et al.  Cultural variability in the effects of question design features on respondent comprehension of health surveys. , 2006, Annals of epidemiology.

[10]  A Hyland,et al.  The conceptual framework of the International Tobacco Control (ITC) Policy Evaluation Project , 2006, Tobacco Control.

[11]  G B Hastings,et al.  Methods of the International Tobacco Control (ITC) Four Country Survey , 2006, Tobacco Control.

[12]  James F. Thrasher,et al.  Evaluación de las políticas contra el tabaquismo en países latinoamericanos en la era del Convenio Marco para el Control del Tabaco , 2006 .

[13]  Patricia L. Goerman.  Adapting Cognitive Interview Techniques for Use in Pretesting Spanish Language Survey Instruments , 2006 .

[14]  Y. Poortinga,et al.  Equivalence of survey data : Relevance for international marketing , 2005 .

[15]  Yuling Pan,et al.  Census Bureau Guideline for the Translation of Data Collection Instruments and Supporting Materials: Documentation on How the Guideline Was Developed , 2005 .

[16]  G. Willis,et al.  Does Pretesting Make a Difference? An Experimental Test , 2004 .

[17]  Tom W. Smith.  Developing and Evaluating Cross‐National Survey Instruments , 2004 .

[18]  B. Pennell,et al.  Survey Questionnaire Translation and Assessment , 2004 .

[19]  J. Smit,et al.  Evaluating survey questions by analysing patterns of behavior codes and question-answer sequences: the diagnostic approach , 2004 .

[20]  Elizabeth Martin,et al.  METHODS FOR TESTING AND EVALUATING SURVEY QUESTIONS , 2004 .

[21]  Peter B. Smith.  Acquiescent Response Bias as an Aspect of Cultural Communication Style , 2004 .

[22]  Kristen Hughes.  Comparing Pretesting Methods: Cognitive Interviews, Respondent Debriefing, and Behavior Coding , 2004 .

[23]  Raydel Valdés Salgado.  [The Framework Convention on Tobacco Control]. , 2003, Salud publica de Mexico.

[24]  F. Vijver,et al.  Cross-Cultural Survey Methods , 2002 .

[25]  F. Vijver,et al.  Bias and equivalence : Cross-cultural perspectives , 2003 .

[26]  Derek Yach,et al.  Framework convention on tobacco controlPossible European linkages , 2000 .

[27]  E. S. Knowles,et al.  Why people say "yes": A dual-process theory of acquiescence. , 1999 .

[28]  Methoden und Analysen Zuma Zentrum für Umfragen,et al.  Cross-cultural survey equivalence , 1998 .

[29]  T. Johnson.  Approaches to equivalence in cross-cultural and cross-national survey research , 1998 .

[30]  F.J.R. van de Vijver,et al.  Methods and Data Analysis for Cross-Cultural Research , 1997 .

[31]  Theresa J. DeMaio,et al.  Cognitive interviewing techniques: In the lab and in the field. , 1996 .

[32]  Floyd J. Fowler,et al.  Improving Survey Questions: Design and Evaluation , 1995 .

[33]  K. Bollen,et al.  MACROCOMPARATIVE RESEARCH METHODS , 1993 .

[34]  Esposito Jl,et al.  Methodologies for evaluating survey questions: some lessons from the redesign of the Current Population Survey. , 1993 .

[35]  E. Somers.  International Agency for Research on Cancer. , 1985, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[36]  Charles F. Cannell,et al.  A Technique for Evaluating Interviewer Performance. , 1976 .

[37]  2 . 2 Developing and assessing comparable questions in cross-cultural survey research on tobacco , 2022 .