Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions.

On the basis of a distinction between test content and method of testing, the present study examined several conceptually and practically important effects relating race, reading comprehension, method of assessment, face validity perceptions, and performance on a situational judgement test using a sample of 241 psychology undergraduates (113 Blacks and 128 Whites). Results showed that the Black-White differences in situational judgment test performance and face validity reactions to the test were substantially smaller in the video-based method of testing than in the paper-and-pencil method. The Race x Method interaction effect on test performance was attributable to differences in reading comprehension and face validity reactions associated with race and method of testing. Implications of the findings were discussed in the context of research on adverse impact and examinee test reactions.

[1]  N. Schmitt,et al.  Reactions to cognitive ability tests: the relationships between race, test performance, face validity perceptions, and test-taking motivation. , 1997, The Journal of applied psychology.

[2]  N. Schmitt,et al.  An Evaluation of Two Strategies for Reducing Adverse Impact and Their Effects on Criterion-Related Validity , 1996 .

[3]  David E. Smith,et al.  THE EFFECTS OF APPLICANTs' REACTIONS TO COGNITIVE ABILITY TESTS AND AN ASSESSMENT CENTER , 1994 .

[4]  A. Dalessio,et al.  predicting insurance agent turnover using a video-based situational judgment test , 1994 .

[5]  Steven D. Maurer,et al.  The validity of employment interviews: A comprehensive review and meta-analysis. , 1994 .

[6]  Steven F. Cronshaw,et al.  Evaluation of video-based assessment in transit operator selection , 1994 .

[7]  Nancy Tippins,et al.  Further studies of the low‐fidelity simulation in the form of a situational inventory , 1993 .

[8]  Karl G. Jöreskog,et al.  Lisrel 8: Structural Equation Modeling With the Simplis Command Language , 1993 .

[9]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[10]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[11]  J. Helms.  Why is there no study of cultural equivalence in standardized cognitive ability testing , 1992 .

[12]  D. B. Matthews.  Learning styles research: Implications for increasing students in teacher education programs. , 1991 .

[13]  Richard D. Arvey,et al.  MOTIVATIONAL COMPONENTS OF TEST TAKING , 1990 .

[14]  M. D. Dunnette,et al.  An alternative selection procedure: The low-fidelity simulation. , 1990 .

[15]  J. H. Steiger.  Structural Model Evaluation and Modification: An Interval Estimation Approach. , 1990, Multivariate behavioral research.

[16]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[17]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[18]  R. P. McDonald,et al.  Structural Equations with Latent Variables , 1989 .

[19]  Lawrence R. James,et al.  Causal modelling in organizational research. , 1989 .

[20]  R. Scott.  Gender and Race Achievement Profiles of Black and White Third-Grade Students , 1987 .

[21]  George C. Thornton,et al.  Meta-analysis of assessment center validity. , 1987 .

[22]  Fritz Drasgow,et al.  Study of the measurement bias of two standardized psychological tests. , 1987 .

[23]  C. Cooper,et al.  International review of industrial and organizational psychology , 1986 .

[24]  Michael P. Kirsch,et al.  METAANALYSES OF VALIDITY STUDIES PUBLISHED BETWEEN 1964 AND 1982 AND THE INVESTIGATION OF STUDY CHARACTERISTICS , 1984 .

[25]  J. Hunter,et al.  Validity and Utility of Alternative Predictors of Job Performance , 1984 .

[26]  F. Drasgow.  Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are the central issues , 1984 .

[27]  D. J. Jackson,et al.  Factor Analysis and Measurement in Sociological Research. , 1983 .

[28]  A. Jensen,et al.  Bias in Mental Testing , 1980 .

[29]  Ivan T. Robertson,et al.  Work sample tests: Validity, adverse impact and applicant reaction , 1982 .

[30]  R. Reilly,et al.  VALIDITY AND FAIRNESS OF SOME ALTERNATIVE EMPLOYEE SELECTION PROCEDURES , 1982 .

[31]  John E. Hunter,et al.  Employment testing: Old theories and new research findings. , 1981 .

[32]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[33]  Wayne F. Cascio,et al.  Applied psychology in personnel management , 1978 .

[34]  John E. Hunter,et al.  Job sample vs. paper-and-pencil trades and technical tests: Adverse impact and examinee attitudes. , 1977 .

[35]  W. Dodd.  CHAPTER 9 – ATTITUDES TOWARD ASSESSMENT CENTER PROGRAMS , 1977 .

[36]  William C. Byham,et al.  Applying the assessment center method , 1977 .

[37]  J. Loehlin,et al.  Race Differences in Intelligence , 1975 .

[38]  D. Sörbom.  A GENERAL METHOD FOR STUDYING DIFFERENCES IN FACTOR MEANS AND FACTOR STRUCTURE BETWEEN GROUPS , 1974 .

[39]  Mary L. Tenopyr.  THE COMPARATIVE VALIDITY OF SELECTED LEADERSHIP SCALES RELATIVE TO SUCCESS IN PRODUCTION MANAGEMENT , 1969 .

[40]  T. Cleary.  TEST BIAS: PREDICTION OF GRADES OF NEGRO AND WHITE STUDENTS IN INTEGRATED COLLEGES , 1968 .

[41]  James I. Brown.  The Nelson-Denny Reading Test. , 1960 .

[42]  D. B. Learner,et al.  A Supervisory Practices Test , 1958 .

[43]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[44]  R. L. Thorndike.  Personnel selection : test and measurement techniques , 1951 .