Selection Testing via the Internet: Practical Considerations and Exploratory Empirical Findings*

Despite a growing body of applied research on using the Internet for some human resource management practices, few studies have provided equivalence information or practical lessons concerning selection testing via the Internet. We identify several issues associated with measurement and validity, the role of several individual characteristics, respondents' reactions and behaviors, and other considerations concerning Internet test administration. We also report results from an exploratory study of the correlation between paper-and-pencil and Internet-administered cognitively oriented selection tests (including timed and untimed, proctored tests). Our empirical results suggest modest degrees of cross-mode equivalence for an untimed situational judgment test (r= .84) and for a timed cognitive ability test (r= .60). Further, some types of items (math, verbal, spatial) in the timed cognitive ability test seem to play a differential role in the reduced cross-mode equivalence. New issues regarding the perception of, and reaction to, items presented via the Internet are presented, and a variety of practical issues are derived and discussed.

[1]  A. Ryan,et al.  Reactions to Computerized Testing in Selection Contexts , 2003 .

[2]  Silvia Moscoso,et al.  Internet-based Personality Testing: Equivalence of Measures and Assesses' Perceptions and Reactions , 2003 .

[3]  Deborah Compeau,et al.  Computer Self-Efficacy: Development of a Measure and Initial Test , 1995, MIS Q..

[4]  J. Comer,et al.  Interactive Effects of Major Response Facilitators , 1990 .

[5]  P. Bobko,et al.  A Model for Predicting Computer Experience from Attitudes Toward Computers , 2001 .

[6]  Mark Snyder,et al.  "To Carve Nature at Its Joints": On the Existence of Discrete Classes in Personality , 1985 .

[7]  Philip Bobko,et al.  The Computer Understanding and Experience Scale: a self-report measure of computer experience , 1998 .

[8]  Paul Rosenfeld,et al.  Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? , 1992 .

[9]  Denise Potosky,et al.  A field study of computer efficacy beliefs as an outcome of training: the role of computer playfulness, computer knowledge, and performance during training , 2002, Comput. Hum. Behav..

[10]  Clarice P. Gressard,et al.  Reliability and Factorial Validity of Computer Attitude Scales , 1984 .

[11]  J. Webster,et al.  The Use of Technologies in the Recruiting, Screening, and Selection Processes for Job Candidates , 2003 .

[12]  F. Drasgow,et al.  Does computerizing paper-and-pencil job attitude scales make a difference? New IRT analyses offer insight. , 2000, The Journal of applied psychology.

[13]  James E. Katz,et al.  A nation of strangers? , 1997, CACM.

[14]  Thomas Hill,et al.  Role of efficacy expectations in predicting the decision to use advanced technologies: The case of computers. , 1987 .

[15]  Joseph J. Martocchio,et al.  Microcomputer playfulness: development of a measure with workplace implications , 1992 .

[16]  F. Vijver,et al.  The incomplete equivalence of the paper-and-pencil and computerized versions of the General Aptitude Test Battery , 1994 .

[17]  N. Anderson.  Applicant and Recruiter Reactions to New Technology in Selection: A Critical Review and Agenda for Future Research , 2003 .

[18]  R. N. Davis,et al.  Web-based administration of a personality questionnaire: Comparison with traditional methods , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[19]  F. Drasgow,et al.  Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. , 1993 .

[20]  Robert L. Mccornack,et al.  A criticism of studies comparing item-weighting methods. , 1956 .

[21]  Robert E. Ployhart,et al.  WEB‐BASED AND PAPER‐AND‐PENCIL TESTING OF APPLICANTS IN A PROCTORED SETTING: ARE PERSONALITY, BIODATA, AND SITUATIONAL JUDGMENT TESTS COMPARABLE? , 2003 .

[22]  Lee Sproull,et al.  Using Electronic Mail for Data Collection in Organizational Research , 1986 .

[23]  John W. Jones,et al.  Technology Trends in Staffing and Assessment: A Practice Note , 2003 .

[24]  T. W. Harrell.  Some history of the Army General Classification Test , 1992 .

[25]  Anne L. Harvey,et al.  The Equivalence of Scores from Automated and Conventional Educational and Psychological Tests: A Review of the Literature. College Board Report No. 88-8. , 1988 .

[26]  John L. Smith,et al.  Using the Internet for psychological research: personality testing on the World Wide Web. , 1999, British journal of psychology.

[27]  E. Surface,et al.  FROM PAPER TO PIXELS: MOVING PERSONNEL SURVEYS TO THE WEB , 2003 .

[28]  M. Mcmanus,et al.  Biodata, Personality, and Demographic Differences of Recruits from Three Sources , 2003 .

[29]  R. LaRose,et al.  Privacy Issues in Internet Surveys , 1999 .

[30]  Jon S. Twing,et al.  The effects of age, gender, and experience on measures of attitude regarding computers , 1991 .

[31]  J. Stanton.  AN EMPIRICAL ASSESSMENT OF DATA COLLECTION USING THE INTERNET , 1998 .

[32]  P. Bobko,et al.  Computer versus paper-and-pencil administration mode and response distortion in noncognitive selection tests. , 1997, The Journal of applied psychology.

[33]  A. Bandura.  Social cognitive theory of self-regulation☆ , 1991 .

[34]  H. Lips,et al.  Gender differences and similarities in attitudes toward computers , 1989 .

[35]  Michael J. Burke,et al.  Computerized Psychological Testing: Overview and Critique , 1987 .

[36]  D. Rubin,et al.  Comparing Correlated but Nonoverlapping Correlations , 1996 .

[37]  Edward W. Miles,et al.  A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pencil measurements: A test of measurement equivalence. , 1995 .

[38]  Karen A. Pasveer,et al.  The making of a personality inventory: Help from the WWW , 1998 .

[39]  Zeki Simsek,et al.  A Primer on Internet Organizational Surveys , 2001 .

[40]  C. Bennett,et al.  Modification of the Minnesota Clerical Test to predict performance on video display terminals. , 1987 .

[41]  Jaeyool Boo,et al.  Computerized and Paper-and-Pencil Versions of the Rosenberg Self-Esteem Scale: A Comparison of Psychometric Features and Respondent Preferences , 2001 .

[42]  Brian C. Cronk,et al.  Personality research on the Internet: A comparison of Web-based and traditional instruments in take-home and in-class settings , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[43]  Jeffrey M. Stanton,et al.  Using Internet/Intranet Web Pages to Collect Organizational Research Data , 2001 .

[44]  Bert F. Green,et al.  Equivalence of Conventional and Computer Presentation of Speed Tests , 1986 .

[45]  Marilyn E. Gist,et al.  EFFECTS OF ALTERNATIVE TRAINING METHODS ON SELF-EFFICACY AND PERFORMANCE IN COMPUTER SOFTWARE TRAINING , 1989 .

[46]  Jane Webster,et al.  The Differential Effects of Software Training Previews on Training Outcomes , 1995 .

[47]  Filip Lievens,et al.  Privacy and Attitudes Towards Internet-Based Selection Systems: A Cross-Cultural Comparison , 2003 .