Options for Conducting Web Surveys

Web surveys can be conducted relatively fast and at relatively low cost. However, Web surveys are often conducted with nonprobability samples and, therefore, a major concern is generalizability. There are two main approaches to address this concern: One, find a way to conduct Web surveys on probability samples without losing most of the cost and speed advantages (e.g., by using mixed-mode approaches or probability-based panel surveys). Two, make adjustments (e.g., propensity scoring, post-stratification, GREG) to nonprobability samples using auxiliary variables. We review both of these approaches as well as lesser-known ones such as respondent-driven sampling. There are many different ways Web surveys can solve the challenge of generalizability. Rather than adopting a one-size-fits-all approach, we conclude that the choice of approach should be commensurate with the purpose of the study.

[1]  Douglas D. Heckathorn,et al.  Respondent-driven sampling : A new approach to the study of hidden populations , 1997 .

[2]  M. Kretzschmar,et al.  Online Respondent-Driven Sampling for Studying Contact Patterns Relevant for the Spread of Close-Contact Pathogens: A Pilot Study in Thailand , 2014, PloS one.

[3]  John Bremer,et al.  The Record of Internet-Based Opinion Polls in Predicting the Results of 72 Races in the November 2000 US Elections , 2001 .

[4]  Kaye Wellings,et al.  Nonprobability Web Surveys to Measure Sexual Behaviors and Attitudes in the General Population: A Comparison With a Probability Sample Interview Survey , 2014, Journal of medical Internet research.

[5]  Mick P Couper,et al.  Issues of representation in eHealth research (with a focus on web surveys). , 2007, American journal of preventive medicine.

[6]  Mick Couper,et al.  Assessment of Innovations in Data Collection Technologyfor Understanding Society , 2012 .

[7]  Vera Toepoel,et al.  Effects of Design in Web Surveys Comparing Trained and Fresh Respondents , 2008 .

[8]  Jelke Bethlehem,et al.  Selection Bias in Web Surveys , 2010 .

[9]  Barry Schouten,et al.  Evaluating Bias of Sequential Mixed-mode Designs Against Benchmark Surveys , 2017 .

[10]  Mario Callegaro,et al.  Report of the Inquiry into the 2015 British general election opinion polls , 2016 .

[11]  Richard Valliant,et al.  Internet Surveys: Can Statistical Adjustments Eliminate Coverage Bias? , 2008 .

[12]  R. Shave,et al.  British adults' views on the health benefits of moderate and vigorous activity. , 2007, Preventive medicine.

[13]  D. Haziza,et al.  Construction of Weights in Surveys: A Review , 2017 .

[14]  M. Elliott,et al.  Composite Estimates from Incomplete and Complete Frames for Minimum-Mse Estimation in a Rare Population An Application to Families with Young Children , 2009 .

[15]  Martin Ortlieb,et al.  A Comparison of Six Sample Providers Regarding Online Privacy Benchmarks , 2014 .

[16]  H. P. Tappan Of the sensitivity. , 1840 .

[17]  D. Yeager,et al.  Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples , 2011 .

[18]  A. Scherpenzeel,et al.  Data Collection in a Probability-Based Internet Panel: How the LISS Panel Was Built and How It Can Be Used , 2011 .

[19]  A. Kapteyn,et al.  Recruiting an Internet Panel Using Respondent-Driven Sampling , 2014 .

[20]  Peter V. Miller,et al.  Web Survey Methods Introduction , 2008 .

[21]  Douglas Rivers,et al.  The 2006 Cooperative Congressional Election Study , 2008 .

[22]  W. G. Cochran The effectiveness of adjustment by subclassification in removing bias in observational studies. , 1968, Biometrics.

[23]  Benjamin M Craig,et al.  Comparison of US Panel Vendors for Online Surveys , 2013, Journal of medical Internet research.

[24]  Linus Bengtsson,et al.  Implementation of Web-Based Respondent-Driven Sampling among Men Who Have Sex with Men in Vietnam , 2012, PloS one.

[25]  Frederick G. Conrad,et al.  Comparisons of Online Recruitment Strategies for Convenience Samples , 2016 .

[26]  Michael R. Elliott,et al.  Inference for Nonprobability Samples , 2017 .

[27]  N. Seeman Use data to challenge mental-health stigma , 2015, Nature.

[28]  J. Bauermeister,et al.  Innovative recruitment using online networks: lessons learned from an online study of alcohol and other drug use utilizing a web-based, respondent-driven sampling (webRDS) strategy. , 2012, Journal of studies on alcohol and drugs.

[29]  A. Blom,et al.  Setting Up an Online Panel Representative of the General Population , 2015 .

[30]  Linus Bengtsson,et al.  The sensitivity of respondent‐driven sampling , 2012 .

[31]  Barry Schouten,et al.  Disentangling mode-specific selection and measurement bias in social surveys. , 2013, Social science research.

[32]  M. Elliott,et al.  Use of a web­based convenience sample to supplement a probability sample , 2007 .

[33]  Anders Holmberg,et al.  Contact strategies to improve participation via the web in a mixed-mode mail and web survey , 2010 .

[34]  Mark S Handcock,et al.  7. Respondent-Driven Sampling: An Assessment of Current Methodology , 2009, Sociological methodology.

[35]  Adalsteinn D. Brown,et al.  World survey of mental illness stigma. , 2016, Journal of affective disorders.

[36]  C. B. Bhutta Not by the Book Facebook as a Sampling Frame , 2012 .

[37]  Matthias Schonlau,et al.  Selection Bias in Web Surveys and the Use of Propensity Scores , 2006 .

[38]  Mario Callegaro,et al.  Online Panel Research: A Data Quality Perspective , 2014 .

[39]  Shalini L Kulasingam,et al.  Estimation of Geographic Variation in Human Papillomavirus Vaccine Uptake in Men and Women: An Online Survey Using Facebook Recruitment , 2014, Journal of medical Internet research.

[40]  M. Elliott,et al.  Conducting Research Surveys Via E-Mail and The Web , 2001 .

[41]  V. Toepoel,et al.  Straightlining in Web survey panels over time , 2015 .

[42]  Rebecca L. Medway,et al.  When More Gets You Less: A Meta-Analysis of the Effect of Concurrent Web Options on Mail Survey Response Rates , 2012 .

[43]  Matthias Schonlau,et al.  Web Surveys as Part of a Mixed-Mode Strategy for Populations that cannot be Contacted by E-Mail , 2003 .

[44]  David M. Rothschild,et al.  Forecasting elections with non-representative polls , 2015 .

[45]  Jon A. Krosnick,et al.  Measuring Intent to Participate and Participation in the 2010 Census and Their Correlates and Trends: Comparisons of RDD Telephone and Non-probability Sample Internet Survey Data , 2010 .

[46]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[47]  Matthias Schonlau,et al.  A Comparison Between Responses From a Propensity-Weighted Web Survey and an Identical RDD Survey , 2004 .

[48]  Cyprian Wejnert,et al.  RESPONDENT-DRIVEN SAMPLING FOR ONLINE RESEARCH , 2007 .

[49]  Douglas Rivers,et al.  Inference From Matched Samples in the 2008 U.S. National Elections , 2009 .

[50]  Sunghee Lee,et al.  Estimation for Volunteer Panel Web Surveys Using Propensity Score Adjustment and Calibration Adjustment , 2009 .

[51]  M. Couper A REVIEW OF ISSUES AND APPROACHES , 2000 .

[52]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[53]  Jelke Bethlehem,et al.  Solving the Nonresponse Problem With Sample Matching? , 2016 .

[54]  Sunghee Lee Propensity score adjustment as a weighting scheme for volunteer panel web surveys , 2006 .

[55]  Vera Toepoel,et al.  Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys , 2011 .

[56]  Frauke Kreuter,et al.  Practical Tools for Designing and Weighting Survey Samples , 2015 .

[57]  Adam J. Berinsky,et al.  Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk , 2012, Political Analysis.

[58]  Douglas Rivers,et al.  Sampling for Web Surveys , 2007, Handbook of Web Surveys.