Do Adjusted Subscores Lack Validity? Don’t Blame the Messenger

There are several techniques that increase the precision of subscores by borrowing information from other parts of the test. These techniques have been criticized on validity grounds in several of the recent publications. In this note, the authors question the argument used in these publications and suggest both inherent limits to the validity argument and empirical issues worth examining.

[1]  Susy Macqueen,et al.  Validity , 1973, Just Algorithms.

[2]  Gautam Puhan,et al.  COMPARISON OF SUBSCORES BASED ON CLASSICAL TEST THEORY METHODS , 2008 .

[3]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[4]  Sandip Sinharay,et al.  How Often Do Subscores Have Added Value? Results from Operational and Simulated Data , 2010 .

[5]  A Bayesian/IRT Index of Objective Performance for Tests with Mixed Item Types 1 , 1997 .

[6]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[7]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .

[8]  Richard M. Luecht,et al.  Applications of Multidimensional Diagnostic Scoring for Certification and Licensure Tests. , 2003 .

[9]  Karee E. Dunn,et al.  A Critical Review of Research on Formative Assessment: The Limited Scientific Evidence of the Impact of Formative Assessment in Education , 2009 .

[10]  Edward M Hall.  Making the Most of What We Have , 1975 .

[11]  Per-Erik Lyrén.  Reporting subscores from college admission tests , 2009 .

[12]  Kathleen M. Sheehan,et al.  Some Paths Toward Making Praxis Scores More Useful , 1998 .

[13]  Howard Wainer,et al.  Augmented Scores-"Borrowing Strength" to Compute Scores Based on Small Numbers ofltems , 2001 .

[14]  Shelby J. Haberman,et al.  When Can Subscores Have Value? , 2008 .

[15]  Shelby J. Haberman,et al.  Reporting of Subscores Using Multidimensional Item Response Theory , 2010 .

[16]  M. Reckase.  The Past and Future of Multidimensional Item Response Theory , 1997 .

[17]  Wendy M. Yen.  A Bayesian/IRT Index of Objective Performance 1 , 1987 .

[18]  Lihua Yao,et al.  A Multidimensional Item Response Modeling Approach for Improving Subscale Proficiency Estimation and Classification , 2007 .

[19]  Shelby J. Haberman.  SUBSCORES AND VALIDITY , 2008 .

[20]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[21]  A Comparison of Approaches for Improving the Reliability of Objective Level Scores , 2010 .

[22]  Identifiers California,et al.  Annual Meeting of the National Council on Measurement in Education , 1998 .

[23]  Clement A. Stone,et al.  Providing Subscale Scores for Diagnostic Information: A Case Study When the Test is Essentially Unidimensional , 2009 .