Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires.

The rapid growth of the Internet provides a wealth of new research opportunities for psychologists. Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N = 361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and age. Moreover, Internet findings generalize across presentation formats, are not adversely affected by nonserious or repeat responders, and are consistent with findings from traditional methods. It is concluded that Internet methods can contribute to many areas of psychology.

[1]  Gary R Lichtenstein.  [Letter to the Editor] , 1996, Nature.

[2]  M. Orne.  On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. , 1962 .

[3]  C. W. Jackson,et al.  Some nondeprivation variables which influence the "effects" of experimental sensory deprivation. , 1966, Journal of abnormal psychology.

[4]  Jackson Cw,et al.  Some nondeprivation variables which influence the "effects" of experimental sensory deprivation. , 1966 .

[5]  S. Jourard,et al.  Disclosing man to himself , 1968 .

[6]  Chris Argyriso.  Issues in Evaluating Laboratory Education , 1968 .

[7]  George A. Miller,et al.  Psychology as a means of promoting human welfare. , 1969 .

[8]  D. Mook,et al.  In defense of external invalidity. , 1983 .

[9]  S. Kiesler,et al.  Response Effects in the Electronic Survey , 1986 .

[10]  D. O. Sears.  College sophomores in the laboratory: Influences of a narrow data base on social psychology's view of human nature. , 1986 .

[11]  S. Levine,et al.  Assessment of suicide risk by computer‐delivered self‐rating questionnaire: preliminary findings , 1989, Acta psychiatrica Scandinavica.

[12]  S. Kiesler,et al.  SELF-SELECTED AND RANDOMLY SELECTED RESPONDENTS IN A COMPUTER NETWORK SURVEY , 1992 .

[13]  L. A. Pervin.  Handbook of Personality: Theory and Research , 1992 .

[14]  Thomas D. Snyder,et al.  Digest of Education Statistics , 1994 .

[15]  A. Feingold,et al.  Gender differences in personality: a meta-analysis. , 1994, Psychological bulletin.

[16]  Shellie D. Locke,et al.  Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats. , 1995 .

[17]  Carl Vogel,et al.  Proper methodologies for psychological and sociological studies conducted via the Internet , 1996, Behavior Research Methods, Instruments, & Computers.

[18]  James M. Kieley,et al.  CGI scripts: Gateways to World-Wide Web power , 1996 .

[19]  Michael A. Smith,et al.  Virtual subjects: Using the Internet as an alternative source of subjects and research environment , 1997 .

[20]  William C. Schmidt,et al.  World-Wide Web survey research: Benefits, potential problems, and solutions , 1997 .

[21]  Robert H. Morrow,et al.  CGI scripts: A strategy for between-subjects experimental group assignment on the World-Wide Web , 1998 .

[22]  Peter F. Merenda,et al.  DEMOGRAPHIC VARIABLES AND PERSONALITY: THE EFFECTS OF GENDER, AGE, EDUCATION, AND ETHNIC/RACIAL STATUS ON SELF-DESCRIPTIONS OF PERSONALITY ATTRIBUTES , 1998 .

[23]  Robert E. Kraut,et al.  Internet paradox. A social technology that reduces social involvement and psychological well-being? , 1998, The American psychologist.

[24]  Karen A. Pasveer,et al.  The making of a personality inventory: Help from the WWW , 1998 .

[25]  O. John,et al.  Los Cinco Grandes across cultures and ethnic groups: multitrait multimethod analyses of the Big Five in Spanish and English. , 1998, Journal of personality and social psychology.

[26]  Attila Szabo,et al.  Guidelines for Internet Research , 1998 .

[27]  Robert D. Tortora,et al.  Principles for Constructing Web Surveys , 1998 .

[28]  S. Gosling,et al.  An integrative approach to personality research in anthrozoology: Ratings of six species of pets and their owners , 1998 .

[29]  S. M. Rogers,et al.  Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. , 1998, Science.

[30]  John L. Smith,et al.  Using the Internet for psychological research: personality testing on the World Wide Web. , 1999, British journal of psychology.

[31]  S. Srivastava,et al.  The Big Five Trait taxonomy: History, measurement, and theoretical perspectives. , 1999 .

[32]  Robert D. Tortora,et al.  INFLUENCE OF PLAIN VS. FANCY DESIGN ON RESPONSE RATES FOR WEB SURVEYS , 1999 .

[33]  Fritz Drasgow,et al.  A Meta-Analytic Study of Social Desirability Distortion in Computer- Administered Questionnaires, Traditional Questionnaires, and Interviews , 1999 .

[34]  M. Birnbaum.  Psychological experiments on the internet , 2000 .

[35]  Oliver P. John,et al.  Measurement: Reliability, construct validation, and scale construction. , 2000 .

[36]  Reeshad S. Dalal,et al.  Validity of Web-Based Psychological Research , 2000 .

[37]  Michael H. Birnbaum,et al.  Introduction to Behavioral Research on the Internet , 2000 .

[38]  K. McGraw,et al.  The Integrity of Web-Delivered Experiments: Can You Trust the Data? , 2000, Psychological science.

[39]  Tom Buchanan,et al.  Potential of the Internet for personality research , 2000 .

[40]  M. Couper.  A REVIEW OF ISSUES AND APPROACHES , 2000 .

[41]  P. Costa,et al.  Gender differences in personality traits across cultures: robust and surprising findings. , 2001, Journal of personality and social psychology.

[42]  Samuel J. Best,et al.  An Assessment of the Generalizability of Internet Surveys , 2001 .

[43]  Thomas D. Snyder,et al.  Digest of Education Statistics 2002 , 2001 .

[44]  Samuel D Gosling,et al.  Global self-esteem across the life span. , 2002, Psychology and aging.

[45]  Jonathon N. Cummings,et al.  Internet Paradox Revisited , 2002 .

[46]  F. A. Pettit.  A comparison of World-Wide Web and paper-and-pencil personality questionnaires , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[47]  Brian A. Nosek,et al.  Harvesting implicit group attitudes and beliefs from a demonstration web site , 2002 .

[48]  S. Gosling,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES The Do Re Mi’s of Everyday Life: The Structure and Personality Correlates of Music Preferences , 2003 .

[49]  Deborah A. Small,et al.  Recent EFFECTS OF FEAR AND ANGER ON PERCEIVED RISKS OF TERRORISMA National Field Experiment , 2015 .

[50]  R. Carnegie.  Psychological Research Online : Opportunities and Challenges , 2003 .

[51]  S. Gosling,et al.  Development of personality in early and middle adulthood: set like plaster or persistent change? , 2003, Journal of personality and social psychology.

[52]  W. Keith Campbell,et al.  Individual differences in narcissism: Inflated self-views across the lifespan and around the world , 2003 .

[53]  S. Gosling,et al.  Personality Processes and Individual Differences E-perceptions: Personality Impressions Based on Personal Websites , 2022 .

[54]  Erin E. Michalak,et al.  Report European Psychologist V Guidelines for Internet Research : An Update , 2004 .

[55]  M. Birnbaum.  Methodological and Ethical Issues in Conducting Social Psychology Research via the Internet. , 2004 .

[56]  Edward G. Sargis,et al.  Social psychological research and the Internet : the promise and peril of a new methodological frontier , 2005 .

[57]  Marcy M. Allen,et al.  The Ever-Shifting Internet Population: A New Look At Internet Access and the Digital Divide, Amanda Lenhart (Ed.). The Pew Internet & American Life Project, Washington, DC (2003) , 2005, Gov. Inf. Q..