Measurement effects between CAPI and Web questionnaires in the UK Household Longitudinal Study

In order to lower costs, the idea has been raised to use a mixed-mode design for the UK Household Longitudinal Study (UKHLS) using Computer-Assisted Personal Interviewing (CAPI) and Web Self-Interviewing (WSI). However, a mixed-mode CAPI-WSI design may damage data-quality because of measurement effects between both modes. Before a decision is made about moving to a mixed-mode design in the UKHLS, measurement effects should be researched in greater detail by identifying questions sensitive to measurement effects, investigating survey design optimization for avoiding measurement effects, and investigating estimation techniques for measurement effects.

[1]  Geert Loosveldt,et al.  Face-to-Face versus Web Surveying in a High-Internet-Coverage Population Differences in Response Quality , 2008 .

[2]  Geert Molenberghs,et al.  Evaluating Mode Effects in Mixed-Mode Survey Data Using Covariate Adjustment Models , 2014 .

[3]  Peter Lugtig,et al.  Estimating Nonresponse Bias and Mode Effects in a Mixed-Mode Survey , 2011 .

[4]  Peter Lynn,et al.  Is it a good idea to optimise question format for mode of data collection? Results from a mixed modes experiment , 2011 .

[5]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[6]  Geert Molenberghs,et al.  Arbitrariness of models for augmented and coarse data, with emphasis on incomplete data and random effects models , 2010 .

[7]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[8]  J. Heckman.  Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations. , 1997 .

[9]  Barry Schouten,et al.  Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions , 2013 .

[10]  Edith D. de Leeuw,et al.  Mixed Mode Surveys: When and Why , 2012 .

[11]  D. Dillman,et al.  International handbook of survey methodology. , 2008 .

[12]  M. Kenward,et al.  Enriched-Data Problems and Essential Non-Identifiability , 2012 .

[13]  Melanie Revilla,et al.  Quality in Unimode and Mixed-Mode designs: A Multitrait-Multimethod approach , 2010 .

[14]  Mick P. Couper,et al.  The Future of Modes of Data Collection , 2011 .

[15]  Geert Loosveldt,et al.  Evaluating Relative Mode Effects in Mixed-Mode Surveys: , 2013 .

[16]  G. Loosveldt.  Face-to-face interviews , 2008 .

[17]  Edith D. de Leeuw,et al.  Choosing the Method of Data Collection , 2008 .

[18]  Dirk Heerwegh,et al.  Mode Differences Between Face-to-Face and Web Surveys: An Experimental Investigation of Data Quality and Social Desirability Effects , 2009 .

[19]  Donald B. Rubin,et al.  Bayesian Inference for Causal Effects: The Role of Randomization , 1978 .

[20]  Norbert Schwarz,et al.  The Psychology of Asking Questions , 2008 .

[21]  Bias and Causation: Models and Judgment for Valid Comparisons , 2010 .

[22]  N. Hatton.  ASKING QUESTIONS , 1979, The Medical journal of Australia.

[23]  Robert D. Tortora,et al.  Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet , 2009 .

[24]  G. Loosveldt,et al.  Measuring Public Opinions About Surveys , 2008 .

[25]  E. Leeuw,et al.  To mix or not to mix data collection modes in surveys. , 2005 .

[26]  Mick P. Couper,et al.  Designing Effective Web Surveys: Preface , 2008 .

[27]  Peter Lynn,et al.  Using Motivational Statements in Web-Instrument Design to Reduce Item-Missing Rates in a Mixed-Mode Context , 2015 .

[28]  G. Molenberghs,et al.  A Method to Evaluate Mode Effects on the Mean and Variance of a Continuous Variable in Mixed‐Mode Surveys , 2012 .

[29]  James J. Heckman,et al.  Randomization as an Instrumental Variable , 1995 .

[30]  Matthias Schonlau,et al.  Advantages and Disadvantages of Internet Research Surveys: Evidence from the Literature , 2002 .

[31]  Geert Molenberghs,et al.  A Method for Evaluating Mode Effects in Mixed-mode Surveys , 2010 .

[32]  Piotr Martin,et al.  A good mix? Mixed mode data collection and cross-national surveys , 2011 .

[33]  Don A. Dillman,et al.  Survey Mode as a Source of Instability in Responses across Surveys , 2005 .

[34]  Michael P. Murray,et al.  Instrumental Variables , 2011, International Encyclopedia of Statistical Science.

[35]  Christopher Winship,et al.  Counterfactuals and Causal Inference: Methods and Principles for Social Research , 2007 .

[36]  J. Pearl.  Causal diagrams for empirical research , 1995 .

[37]  Paul Biemer,et al.  Nonresponse Bias and Measurement Bias in a Comparison of Face to Face and Telephone Interviewing , 2001 .

[38]  Jolene D. Smyth,et al.  Comparing Check-All and Forced-Choice Question Formats in Web Surveys , 2006 .

[39]  J. Krosnick,et al.  AN EVALUATION OF A COGNITIVE THEORY OF RESPONSE-ORDER EFFECTS IN SURVEY MEASUREMENT , 1987 .

[40]  Peter M. Steiner,et al.  Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments , 2008 .

[41]  Angela P. Wetzel.  Internet, mail, and mixed‐mode surveys: The tailored design method , 2010 .

[42]  Melanie C. Green,et al.  Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias , 2003 .

[43]  J. Krosnick.  Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[44]  Jolene D. Smyth,et al.  Does “Yes or No” on the Telephone Mean the Same as “Check-All-That-Apply” on the Web? , 2008 .

[45]  D. Katz.  The American Statistical Association , 2000 .

[46]  Peter Lynn,et al.  Assessing the Effect of Data Collection Mode on Measurement , 2010 .

[47]  G. Loosveldt,et al.  Assessing Mode Effects in a National Crime Victimization Survey using Structural Equation Models: Social Desirability Bias and Acquiescence , 2011 .