Web-based research: Methodological variables’ effects on dropout and sample characteristics

Previous research has made a beginning in addressing the importance of methodological differences in Web-based research. The present paper presents four studies investigating whether sample type, financial incentives, time when personal information is requested, table design, and method of obtaining informed consent influence dropout and sample characteristics(both demographics and measured attitudes). Undergraduates were less likely to drop out than nonstudents, and nonstudents offered a financial incentive were less likely to drop out than those offered no incentive. Complex tables, tables that were too wide, requests for personal information on the first page, and the imposing of additional informed consent procedures each provoked early dropout. As was expected, nonstudents and those presented with complex tables showed more measurement error and attitude differences. Asking for personal information and imposing additional consent procedures affected the demographic makeup, raising challenges to external validity.

[1]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[2]  William Oakes,et al.  External validity and the use of real people as subjects. , 1972 .

[3]  Milton D. Hakel,et al.  Attitudes toward Surveys: Development of a Measure and Its Relationship to Respondent Behavior , 2001 .

[4]  H. Levene.  Robust tests for equality of variances , 1961 .

[5]  Mark A. Geistfeld.  Reconciling Cost-Benefit Analysis with the Principle that Safety Matters More than Money , 2001 .

[6]  Ulf-Dietrich Reips.  Chapter 4 – The Web Experiment Method: Advantages, Disadvantages, and Solutions , 2000 .

[7]  E. Singer,et al.  Leverage-saliency theory of survey participation: description and an illustration. , 2000, Public opinion quarterly.

[8]  Donald A. Hantula,et al.  The effects of download delay on performance and end-user satisfaction in an Internet tutorial , 2001, Comput. Hum. Behav..

[9]  James S. Phillips,et al.  Subject Recruitment Bias: The Paid Volunteer Subject , 1978 .

[10]  Dara V. O'Neil.  Analysis of Internet Users’ Level of Online Privacy Concerns , 2001 .

[11]  Ulf-Dietrich Reips.  Standards for Internet-based experimenting. , 2002 .

[12]  Ulf-Dietrick Reips,et al.  Internet-Based Psychological Experimenting , 2002 .

[13]  Ulf-Dietrich Reips.  Standards for Internet-based experimenting. , 2002, Experimental psychology.

[14]  Robert M. Groves,et al.  UNDERSTANDING THE DECISION TO PARTICIPATE IN A SURVEY , 1992 .

[15]  S. Penrod,et al.  EXPLORING THE EFFECTS OF ATTITUDES TOWARD THE DEATH PENALTY ON CAPITAL SENTENCING VERDICTS , 2004 .

[16]  J. Krosnick,et al.  Survey research. , 1999, Annual review of psychology.

[17]  Ulf-Dietrich Reips,et al.  A Brief History of Web Experimenting , 2000 .

[18]  D. O. Sears.  College sophomores in the laboratory: Influences of a narrow data base on social psychology's view of human nature. , 1986 .

[19]  Ulf-Dietrich Reips,et al.  The Web Experiment Method , 2000 .

[20]  Steven D. Penrod,et al.  Methodological variables in Web-based research that may affect results: Sample type, monetary incentives, and personal information , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[21]  Mick P. Couper,et al.  THE IMPACT OF PRIVACY AND CONFIDENTIALITY CONCERNS ON SURVEY PARTICIPATION THE CASE OF THE 1990 U.S. CENSUS , 1993 .

[22]  Michael A. Smith,et al.  Virtual subjects: Using the Internet as an alternative source of subjects and research environment , 1997 .

[23]  Ulf-Dietrich Reips,et al.  Financial Incentives, Personal Information and Drop Out in Online Studies , 2001 .