The Magnitude and Extent of Cheating and Response Distortion Effects on Unproctored Internet-Based Tests of Cognitive Ability and Personality

The use of unproctored internet-based testing (UIT) for employee selection is quite widespread. Although this mode of testing has advantages over onsite testing, researchers and practitioners continue to be concerned about potential malfeasance (e.g., cheating and response distortion) under high-stakes conditions. Therefore, the primary objective of the present study was to investigate the magnitude and extent of high- and low-stakes retest effects on the scores of a UIT speeded cognitive ability test and two UIT personality measures. These data permitted inferences about the magnitude and extent of malfeasant responding. The study objectives were accomplished by implementing two within-subjects design studies (Study 1 N ¼296; Study 2 N ¼318) in which test takers first completed the tests as job applicants (high-stakes) or incumbents (low-stakes) then as research participants (low-stakes). For the speeded cognitive ability measure, the pattern of test score differences was more consonant with a psychometric practice effect than a malfeasance explanation. This result is likely due to the speeded nature of the test. And for the UIT personality measures, the pattern of higher high-stakes scores compared with lower low-stakes scores is similar to those reported for proctored tests in the extant literature. Thus, our results indicate that the use of a UIT administration does not uniquely threaten personality measures in terms of elevated scores under high-stakes testing that are higher than those observed for proctored tests in the extant literature.

[1]  Richard L. Griffith,et al.  Do applicants fake? An examination of the frequency of applicant faking behavior , 2007 .

[2]  N. Tippins.  Internet Alternatives to Traditional Proctored Testing: Where Are We Now? , 2009, Industrial and Organizational Psychology.

[3]  David E. Wiley,et al.  Socially Desirable Responding: The Evolution of a Construct , 2001 .

[4]  Frederick L. Oswald,et al.  Personality Testing and Industrial–Organizational Psychology: Reflections, Progress, and Prospects , 2008, Industrial and Organizational Psychology.

[5]  Gregory J. Cizek,et al.  Cheating on Tests : How To Do It, Detect It, and Prevent It , 1999 .

[6]  Fritz Drasgow,et al.  UNPROCTORED INTERNET TESTING IN EMPLOYMENT SETTINGS , 2006 .

[7]  Kenneth Pearlman.  Unproctored Internet Testing: Practical, Legal, and Ethical Concerns , 2009, Industrial and Organizational Psychology.

[8]  H. Gulliksen.  Theory of mental tests , 1952 .

[9]  Fritz Drasgow,et al.  A Meta-Analytic Study of Social Desirability Distortion in Computer- Administered Questionnaires, Traditional Questionnaires, and Interviews , 1999 .

[10]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[11]  David E. Wiley,et al.  The role of constructs in psychological and educational measurement , 2001 .

[12]  Michael T. Brannick,et al.  A Meta-Analytic Investigation of Job Applicant Faking on Personality Measures , 2006 .

[13]  Jill E. Ellingson,et al.  Personality assessment across selection and development contexts: insights into response distortion. , 2007, The Journal of applied psychology.

[14]  Newell K. Eaton,et al.  Criterion-related validities of personality constructs and the effect of response distortion on those validities , 1990 .

[15]  W. S. Zimmerman,et al.  The Guilford-Zimmerman temperament survey handbook : twenty-five years of research and application , 1976 .

[16]  John P. Hausknecht,et al.  Retesting in selection: a meta-analysis of coaching and practice effects for tests of cognitive ability. , 2007, The Journal of applied psychology.

[17]  Fritz Drasgow,et al.  Two-Step Testing in Employee Selection: Is Score Inflation a Problem? , 2008 .

[18]  P. Bobko,et al.  Computer versus paper-and-pencil administration mode and response distortion in noncognitive selection tests. , 1997, The Journal of applied psychology.

[19]  Philip Bobko,et al.  Selection Testing via the Internet: Practical Considerations and Exploratory Empirical Findings* , 2004 .

[20]  Marise Ph. Born,et al.  The Role of Constructs in Psychological and Educational Measurement , 2002 .

[21]  F. J. Dudek.  The Continuing Misinterpretation of the Standard Error of Measurement , 1979 .

[22]  Chockalingam Viswesvaran,et al.  Meta-Analyses of Fakability Estimates: Implications for Personality Measurement , 1999 .

[23]  L. Hough.  Effects of Intentional Distortion in Personality Measurement and evaluation of Suggested Palliatives , 1998 .

[24]  Joyce Hogan,et al.  Personality measurement, faking, and employment selection. , 2007, The Journal of applied psychology.

[25]  N. Schmitt,et al.  The impact of corrections for faking on the validity of noncognitive measures in selection settings. , 2006, The Journal of applied psychology.

[26]  B. Whitley,et al.  FACTORS ASSOCIATED WITH CHEATING AMONG COLLEGE STUDENTS: A Review , 1998 .

[27]  Paul Rosenfeld,et al.  Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? , 1992 .

[28]  Murray R. Barrick,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS , 1991 .

[29]  Chockalingam Viswesvaran,et al.  Role of social desirability in personality testing for personnel selection: The red herring. , 1996 .

[30]  Robert E. Ployhart,et al.  WEB‐BASED AND PAPER‐AND‐PENCIL TESTING OF APPLICANTS IN A PROCTORED SETTING: ARE PERSONALITY, BIODATA, AND SITUATIONAL JUDGMENT TESTS COMPARABLE? , 2003 .

[31]  Eugene Burke,et al.  Preserving the Integrity of Online Testing , 2009, Industrial and Organizational Psychology.

[32]  Christopher L. Martin,et al.  Socially Desirable Responding in Computerized Questionnaires: When Questionnaire Purpose Matters More Than the Mode1 , 2002 .