Predicting Performance in Higher Education Using Proximal Predictors

We studied the validity of two methods for predicting academic performance and student-program fit that were proximal to important study criteria. Applicants to an undergraduate psychology program participated in a selection procedure containing a trial-studying test based on a work sample approach, and specific skills tests in English and math. Test scores were used to predict academic achievement and progress after the first year, achievement in specific course types, enrollment, and dropout after the first year. All tests showed positive significant correlations with the criteria. The trial-studying test was consistently the best predictor in the admission procedure. We found no significant differences between the predictive validity of the trial-studying test and prior educational performance, and substantial shared explained variance between the two predictors. Only applicants with lower trial-studying scores were significantly less likely to enroll in the program. In conclusion, the trial-studying test yielded predictive validities similar to that of prior educational performance and possibly enabled self-selection. In admissions aimed at student-program fit, or in admissions in which past educational performance is difficult to use, a trial-studying test is a good instrument to predict academic performance.

[1]  G. Norman,et al.  The Ability of the Multiple Mini-Interview to Predict Preclerkship Performance in Medical School , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[2]  F. Schmidt,et al.  The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. , 1998 .

[3]  Self‐appraisal: A convergence of selection and guidance , 1978 .

[4]  R. Meijer,et al.  Selection of medical students on the basis of non-academic skills: is it worth the trouble? , 2016, Clinical medicine.

[5]  G. Norman,et al.  Multiple mini‐interviews predict clerkship and licensing examination performance , 2007, Medical education.

[6]  B. de Raad,et al.  Personality in learning and education: a review , 1996 .

[7]  G. C. Thornton,et al.  Work sample tests. , 2013 .

[8]  Sarah A. Hezlett,et al.  A comprehensive meta-analysis of the predictive validity of the graduate record examinations: implications for graduate student selection and performance. , 2001, Psychological bulletin.

[9]  J. Breaugh.  Employee recruitment: Current knowledge and important areas for future research , 2008 .

[10]  G. Norman,et al.  Predictive validity of the multiple mini‐interview for selecting medical trainees , 2009, Medical education.

[11]  Sarah A. Hezlett,et al.  Standardized Tests Predict Graduate Students' Success , 2007, Science.

[12]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[13]  S. Zedeck,et al.  Admission to Law School: New Measures , 2012 .

[14]  Janet F. Carlson,et al.  APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology. , 2013 .

[15]  Filip Lievens,et al.  Adjusting medical school admission: assessing interpersonal skills using situational judgement tests , 2013, Medical education.

[16]  Ute R. Hülsheger,et al.  Applicant Reactions in Selection: Comprehensive Meta-Analysis into Reaction Generalization Versus Situational Specificity , 2010 .

[17]  R. Moos,et al.  The generality of specificity. , 1967, Archives of general psychiatry.

[18]  I. Häkkinen.  Essays on School Resources, Academic Achievement and Student Employment , 2004 .

[19]  J. Cohen-schotanus,et al.  Selection and study performance: comparing three admission processes within one medical school , 2014, Medical education.

[20]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[21]  J. H. Steiger.  Tests for comparing elements of a correlation matrix. , 1980 .

[22]  James J. Asher,et al.  REALISTIC WORK SAMPLE TESTS: A REVIEW , 1974 .

[23]  Ivan T. Robertson,et al.  Work Sample Testing , 2000 .

[24]  Richard C. Atkinson,et al.  Reflections on a Century of College Admissions Tests , 2009 .

[25]  Filip Lievens,et al.  Situational Tests in Student Selection: An Examination of Predictive Validity, Adverse Impact, and Construct Validity , 2002 .

[26]  I. Peers,et al.  Influence of learning context on the relationship between A-level attainment and final degree performance: a meta-analytic review , 1994 .

[27]  D. Bacon,et al.  GPA in Research Studies: An Invaluable but Neglected Opportunity , 2006 .

[28]  J. Hox.  Multilevel analysis: Techniques and applications, 2nd ed. , 2010 .

[29]  N. Schmitt.  Development of Rationale and Measures of Noncognitive College Student Potential , 2012 .

[30]  Robert E. Ployhart,et al.  Staffing Organizations: Contemporary Practice and Theory , 2005 .

[31]  Ute R. Hülsheger,et al.  Graduate Student Selection: Graduate Record Examination, Socioeconomic Status, and Undergraduate Grade Point Average as Predictors of Study Success in a Western European University , 2015 .

[32]  T. Stijnen,et al.  Academic and non-academic selection criteria in predicting medical school performance , 2013, Medical teacher.

[33]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[34]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[35]  P. F. Wernimont,et al.  Signs, samples, and criteria. , 1968, The Journal of applied psychology.