Tier One Performance Screen Initial Operational Test and Evaluation: Early Results

Abstract : Along with educational, medical, and moral screens, the U.S. Army uses a composite score from the Armed Services Vocational Aptitude Battery (ASVAB), the Armed Forces Qualification Test (AFQT) to select new Soldiers. Although the AFQT is useful for selecting new Soldiers, other personal attributes are important to Soldier performance and retention. Based on the U.S. Army Research Institute's (ARI) investigations, the Army selected one promising measure, the Tailored Adaptive Personality Assessment System (TAPAS), for an initial operational test and evaluation (IOT&E), beginning administration to applicants in 2009. Criterion data are being collected at 6-month intervals from administrative records, from Initial Military Training (IMT), and from schools for eight military occupational specialties (MOS) and will be followed by two waves of data collection from Soldiers at first unit of assignment. This is the first of six planned evaluations of the IOT&E. This report documents the early analyses from a small sample of Soldiers who completed the TAPAS and completed IMT. Similar to prior experimental research, our early evaluation suggests that several TAPAS scales significantly predicted a number of criteria of interest, indicating that the measure holds promise for both selection and classification purposes.

[1]  Leonard A. White,et al.  Lessons Learned in Transitioning Personality Measures From Research to Operational Settings , 2008, Industrial and Organizational Psychology.

[2]  Fritz Drasgow,et al.  Examining assumptions about item responding in personality assessment: should ideal point methods be considered for scale development and scoring? , 2006, The Journal of applied psychology.

[3]  Deirdre J Knapp,et al.  Army Enlisted Personnel Competency Assessment Program: Phase 2 Report , 2006 .

[4]  F. Drasgow,et al.  An IRT Approach to Constructing and Scoring Pairwise Preference Items Involving Stimuli on Different Dimensions: The Multi-Unidimensional Pairwise-Preference Model , 2005 .

[5]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[6]  Deirdre J. Knapp,et al.  Concurrent Validation of Experimental Army Enlisted Personnel Selection and Classification Measures , 2007 .

[7]  W. D. Corte.  Estimating the classification efficiency of a test battery , 2000 .

[8]  Deirdre J Knapp,et al.  Expanded Enlistment Eligibility Metrics (EEEM): Recommendations on a Non-Cognitive Screen for New Soldier Selection , 2010 .

[9]  Christopher E. Sager,et al.  Development of Experimental Army Enlisted Personnel Selection and Classification Tests and Job Performance Criteria , 2005 .

[10]  John P. Campbell,et al.  MODELING JOB PERFORMANCE IN A POPULATION OF JOBS , 1990 .

[11]  Deirdre J. Knapp,et al.  Validating Future Force Performance Measures (Army Class): In-Unit Performance Longitudinal Validation , 2012 .

[12]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[13]  Stephen Stark,et al.  Adaptive Testing With the Multi-Unidimensional Pairwise Preference Model , 2007 .

[14]  Tirso E. Diaz,et al.  Ill-structured measurement designs in organizational research: implications for estimating interrater reliability. , 2008, The Journal of applied psychology.

[15]  John P. Campbell,et al.  Exploring the Limits in Personnel Selection and Classification , 2001 .

[16]  J. S. Roberts,et al.  A General Item Response Theory Model for Unfolding Unidimensional Polytomous Responses , 2000 .

[17]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[18]  Dan J. Putka,et al.  Investigations into Army Enlisted Classification Systems: Concurrent Validation Report , 2009 .

[19]  S. Stark.  A New IRT Approach to Test Construction and Scoring Designed to Reduce the Effects of Faking in Personality Assessment: The Generalized Graded Unfolding Model for Multi -Unidimensional Paired Comparison Responses , 2002 .

[20]  J. Shane.  Analysis and Findings , 2013 .

[21]  William J Strickland.  A Longitudinal Examination of First Term Attrition and Reenlistment among FY1999 Enlisted Accessions , 2005 .

[22]  Deirdre J Knapp,et al.  Validating Future Force Performance Measures (Army Class): Reclassification Test and Criterion Development , 2009 .

[23]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[24]  Paul Horst,et al.  A technique for the development of a differential prediction battery. , 1954 .

[25]  Deirdre J Knapp,et al.  Validating Future Force Performance Measures (Army Class): End of Training Longitudinal Validation , 2009 .

[26]  Fritz Drasgow,et al.  Constructing personality scales under the assumptions of an ideal point response process: toward increasing the flexibility of personality measures. , 2007, Psychological assessment.