Human research and data collection via the internet.

Advantages and disadvantages of Web and lab research are reviewed. Via the World Wide Web, one can efficiently recruit large, heterogeneous samples quickly, recruit specialized samples (people with rare characteristics), and standardize procedures, making studies easy to replicate. Alternative programming techniques (procedures for data collection) are compared, including client-side as opposed to server-side programming. Web studies have methodological problems; for example, higher rates of drop out and of repeated participation. Web studies must be thoroughly analyzed and tested before launching on-line. Many studies compared data obtained in Web versus lab. These two methods usually reach the same conclusions; however, there are significant differences between college students tested in the lab and people recruited and tested via the Internet. Reasons that Web researchers are enthusiastic about the potential of the new methods are discussed.

[1]  M H Birnbaum,et al.  SurveyWiz and factorWiz: Javascript Web pages that make HTML forms for research on the internet , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[2]  Robert H. Morrow,et al.  CGI scripts: A strategy for between-subjects experimental group assignment on the World-Wide Web , 1998 .

[3]  Adam Joinson,et al.  Social desirability, anonymity, and internet-based questionnaires , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[4]  Michael H. Birnbaum,et al.  How to show that 9 > 221 : Collect judgments in a between-subjects design , 1999 .

[5]  Aimée M. Surprenant,et al.  The Cognitive Psychology Online Laboratory , 2000 .

[6]  John Mueller,et al.  What Are Computing Experiences Good For?: A Case Study in Online Research , 2000 .

[7]  Ulf-Dietrich Reips,et al.  A Brief History of Web Experimenting , 2000 .

[8]  Ulf-Dietrich Reips,et al.  The Web Experiment Method , 2000 .

[9]  John L. Smith,et al.  Using the Internet for psychological research: personality testing on the World Wide Web. , 1999, British journal of psychology.

[10]  Michael H Birnbaum,et al.  Web-based experiments controlled by JavaScript: An example from probability learning , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[11]  Ulf-Dietrich Reips,et al.  Das psychologische Experimentieren im Internet , 1997 .

[12]  M S Horswill,et al.  User-controlled photographic animations, photograph-based questions, and questionnaires: Three Internet-based instruments for measuring drivers’ risk-taking behavior , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[13]  William C. Schmidt,et al.  The Server Side of Psychology Web Experiments , 2000 .

[14]  M. Birnbaum.  Psychological experiments on the internet , 2000 .

[15]  W C Schmidt.  Presentation accuracy of Web animation methods , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[16]  M. Birnbaum.  Decision Making in the Lab and on the Web , 2000 .

[17]  Katelyn Y. A. McKenna,et al.  The internet and social life. , 2004, Annual review of psychology.

[18]  Mark D. Tew,et al.  Undergraduate labs and computers: The case for PsychExps , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[19]  M. Birnbaum,et al.  Emerging Perspectives on Judgment and Decision Research: Generalization Across People, Procedures, and Predictions: Violations of Stochastic Dominance and Coalescing , 2003 .

[20]  Ron Hoffman,et al.  Operate your own World-Wide Web server , 1997 .

[21]  Diane J. Schiano,et al.  Convergent methodologies in cyber-psychology: A case study , 1997 .

[22]  Steven D. Penrod,et al.  Methodological variables in Web-based research that may affect results: Sample type, monetary incentives, and personal information , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[23]  Ulf-Dietrich Reips.  Standards for Internet-based experimenting. , 2002, Experimental psychology.

[24]  Robert E. Kraut,et al.  Internet paradox. A social technology that reduces social involvement and psychological well-being? , 1998, The American psychologist.

[25]  John L. Smith,et al.  Research on the Internet: Validation of a World-Wide Web mediated personality scale , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[26]  Arndt Bröder,et al.  Improving survey research on the World-Wide Web using the randomized response technique , 2001 .

[27]  Carl Vogel,et al.  Proper methodologies for psychological and sociological studies conducted via the Internet , 1996, Behavior Research Methods, Instruments, & Computers.

[28]  Heinrich H. Bülthoff,et al.  Psychophysics tests on the internet: an evaluation , 1998 .

[29]  J. Gackenbach.  Psychology and the Internet : intrapersonal, interpersonal, and transpersonal implications , 1998 .

[30]  Ulf-Dietrich Reips,et al.  WEXTOR: A Web-based tool for generating and visualizing experimental designs and procedures , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[31]  Tom Buchanan,et al.  Online personality assessment , 2001 .

[32]  M. Birnbaum.  Testing Critical Properties of Decision Making on the Internet , 1999 .

[33]  Patricia Wallace.  The Psychology of the Internet , 1999 .

[34]  Helmut Krueger,et al.  A tutorial on creating logfiles for event-driven applications , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[35]  Michael Bosnjak,et al.  Participation in Non-Restricted Web-Surveys: A Typology and Explanatory Model for Item-Nonresponse , 2001 .

[36]  M Lange.  Museum of perception and cognition website: Using JavaScript to increase interactivity in Web-based presentations , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[37]  J Eichstaedt,et al.  An inaccurate-timing filter for reaction time measurement by JAVA applets implementing Internet-based experiments , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[38]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[39]  Ulf-Dietrich Reips,et al.  Merging Field and Institution : Running a Web Laboratory , 2001 .

[40]  Ulf-Dietrich Reips,et al.  Theory and Techniques of Conducting Web Experiments , 2002 .

[41]  Reeshad S. Dalal,et al.  Validity of Web-Based Psychological Research , 2000 .

[42]  Jochen Musch,et al.  Psychological experimenting on the World-Wide Web : Investigating content effects in syllogistic reasoning , 2002 .

[43]  U D Reips.  The Web Experimental Psychology Lab: Five years of data collection on the Internet , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[44]  Michael H. Birnbaum,et al.  Introduction to Behavioral Research on the Internet , 2000 .

[45]  Ulf-Dietrick Reips,et al.  Internet-Based Psychological Experimenting , 2002 .

[46]  R J White,et al.  Quiz-o-Matic: A free Web-based tool for construction of self-scoring on-line quizzes , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[47]  M. Birnbaum.  A Web-Based Program of Research on Decision Making , 2001 .

[48]  H Hecht,et al.  A perception experiment with time-critical graphics animation on the World-Wide Web , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[49]  Karen A. Pasveer,et al.  The making of a personality inventory: Help from the WWW , 1998 .

[50]  M. Birnbaum.  Methodological and Ethical Issues in Conducting Social Psychology Research via the Internet. , 2004 .

[51]  J. Eichstaedt,et al.  Measuring differences in preactivation on the Internet: the content category superiority effect. , 2002, Experimental psychology.

[52]  Jonathan Baron,et al.  Techniques for Creating and Using Web Questionnaires in Research and Teaching , 2000 .

[53]  Alan Schwartz,et al.  Tutorial: Perl, a psychologically efficient reformatting language , 1998 .

[54]  Norma Welch,et al.  The World-Wide Web as a medium for psychoacoustical demonstrations and experiments: Experience and results , 1996 .

[55]  A K Stevenson,et al.  Java experiments for introductory cognitive psychology courses , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[56]  Roger Tourangeau,et al.  Survey research and societal change. , 2004, Annual review of psychology.

[57]  Ulf-Dietrich Reips,et al.  Financial Incentives, Personal Information and Drop Out in Online Studies , 2001 .

[58]  John H. Krantz,et al.  Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness , 1997 .

[59]  Michael A. Smith,et al.  Virtual subjects: Using the Internet as an alternative source of subjects and research environment , 1997 .

[60]  Ulf-Dietrich Reips,et al.  Dimensions of Internet science , 2001 .

[61]  W. Schmidt,et al.  World-Wide Web survey research made easy with WWW Survey Assistant , 1997 .

[62]  Ulf-Dietrich Reips,et al.  Context Effects in Web Surveys , 2002 .

[63]  K. McGraw,et al.  The Integrity of Web-Delivered Experiments: Can You Trust the Data? , 2000, Psychological science.

[64]  Katelyn Y. A. McKenna,et al.  Plan 9 From Cyberspace: The Implications of the Internet for Personality and Social Psychology , 2000 .

[65]  Tom Buchanan,et al.  Potential of the Internet for personality research , 2000 .

[66]  Michael H. Birnbaum,et al.  Mediated Models for the Analysis of Confounded Variables and Self-Selected Samples , 1989 .

[67]  L. Parkinson,et al.  A comparison of four computer-based telephone interviewing methods: Getting answers to sensitive questions , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[68]  William C. Schmidt,et al.  World-Wide Web survey research: Benefits, potential problems, and solutions , 1997 .

[69]  Rüdiger F Pohl,et al.  Hindsight bias around the world. , 2002, Experimental psychology.

[70]  J. Stanton.  AN EMPIRICAL ASSESSMENT OF DATA COLLECTION USING THE INTERNET , 1998 .

[71]  Alison I. Piper,et al.  Conducting Social Science Laboratory Experiments on the World Wide Web. , 1998 .

[72]  Stefan Schwarz,et al.  CGI versus JavaScript : a Web Experiment on the Reversed Hindsight Bias , 2001 .

[73]  Tracy L. Tuten,et al.  Internet Surveys and Data Quality : A Review , 2002 .

[74]  A. Joinson.  Understanding the Psychology of Internet Behaviour: Virtual Worlds, Real Lives , 2002 .

[75]  Mark D. Tew,et al.  PsychExps: An Online Psychology Laboratory , 2000 .

[76]  Luigi Lombardi,et al.  An Intercultural Examination of Facial Features Communicating Surprise , 2000 .

[77]  F. Langerak,et al.  Non-Probability Sampling for WWW Surveys: A Comparison of Methods , 1998 .

[78]  Robert D. Bailey.  Chapter 6 – Human Sexual Behavior: A Comparison of College and Internet Surveys , 2000 .

[79]  H. Herne,et al.  How to Lie with Statistics , 1973 .