Item Selection and Hypothesis Testing for the Adaptive Measurement of Change

Assessing individual change is an important topic in both psychological and educational measurement. An adaptive measurement of change (AMC) method had previously been shown to exhibit greater efficiency in detecting change than conventional nonadaptive methods. However, little work had been done to compare different procedures within the AMC framework. This study introduced a new item selection criterion and two new test statistics for detecting change with AMC that were specifically designed for the paradigm of hypothesis testing. In two simulation sets, the new methods for detecting significant change improved on existing procedures by demonstrating better adherence to Type I error rates and substantially better power for detecting relatively small change.

[1]  E. Muraki.  A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[2]  Sharon L. Weinberg,et al.  Practical Guidelines in Applying Current Theories to the Measurement of Change , 1975 .

[3]  R. P. McDonald,et al.  Test Theory: A Unified Treatment , 1999 .

[4]  F. Baker,et al.  Item response theory : parameter estimation techniques , 1993 .

[5]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[6]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[7]  Erling B. Andersen,et al.  Estimating latent correlations between repeated testings , 1985 .

[8]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[9]  Quinn McNemar,et al.  On Growth Measurement , 1958 .

[10]  David J. Weiss,et al.  A Comparison of IRT-Based Adaptive Mastery Testing and a Sequential Mastery Testing Procedure , 1983 .

[11]  Z. Ying,et al.  a-Stratified Multistage Computerized Adaptive Testing with b Blocking , 2001 .

[12]  Kevin Talbot,et al.  Measurement of change , 2009 .

[13]  Hua-Hua Chang,et al.  The asymptotic posterior normality of the latent trait in an IRT model , 1993 .

[14]  Linda M. Collins,et al.  New methods for the analysis of change , 2001 .

[15]  John B. Willett,et al.  Measuring change: What individual growth modeling buys you. , 1997 .

[16]  Cornelis A.W. Glas,et al.  Computerized adaptive testing : theory and practice , 2000 .

[17]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[18]  van Paul Geert,et al.  Change and development: Issues of theory, method and application , 1997 .

[19]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[20]  David J. Weiss,et al.  Book Review : New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing David J. Weiss (Ed.) New York: Academic Press, 1983, 345 pp., $35.00 , 1984 .

[21]  Keith F. Widaman,et al.  New Methods for the Analysis of Change , 2003 .

[22]  R. Schiffer,et al.  INTRODUCTION , 1988, Neurology.

[23]  Jeffery A. Burr,et al.  Chapter 1 – Change Measurement* , 1990 .

[24]  Susan E. Embretson,et al.  A multidimensional latent trait model for measuring learning and change , 1991 .

[25]  Martha L. Stocking,et al.  Controlling Item Exposure Conditional on Ability in Computerized Adaptive Testing , 1995 .

[26]  David J. Weiss,et al.  Comparison of Computerized Adaptive Testing and Classical Methods for Measuring Individual Change , 2007 .

[27]  Tze Leung Lai,et al.  Power, sample size and adaptation considerations in the design of group sequential clinical trials , 2003 .

[28]  Jay Bartroff,et al.  Modern Sequential Analysis and Its Applications to Computerized Adaptive Testing , 2011, 1106.2559.

[29]  Donald Hedeker,et al.  Statistical Methods in Longitudinal Research:@@@(Vol. I) Principles and Structuring Change@@@(Vol. II) Time Series and Categorical Longitudinal Data. , 1991 .

[30]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[31]  Thomas Knapp,et al.  The Measurement of Change , 1998 .

[32]  Vern W. Urry,et al.  TAILORED TESTING: A SUCCESSFUL APPLICATION OF LATENT TRAIT THEORY* , 1977 .

[33]  A. Agresti.  An introduction to categorical data analysis , 1997 .

[34]  Susan E. Embretson,et al.  Implications of a multidimensional latent trait model for measuring change. , 1991 .

[35]  Wendy M. Yen,et al.  THE CHOICE OF SCALE FOR EDUCATIONAL MEASUREMENT: AN IRT PERSPECTIVE , 1986 .

[36]  Gyenam Kim-Kang,et al.  Adaptive Measurement of Individual Change , 2008 .

[37]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[38]  David J. Weiss,et al.  APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS , 1984 .

[39]  Anthony R. Zara,et al.  Procedures for Selecting Items for Computerized Adaptive Tests. , 1989 .

[40]  Theodorus Johannes Hendrikus Maria Eggen,et al.  Item Selection in Adaptive Testing with the Sequential Probability Ratio Test , 1999 .

[41]  D. Siegmund.  Sequential Analysis: Tests and Confidence Intervals , 1985 .

[42]  L. Tucker,et al.  A base-free measure of change. , 1966, Psychometrika.

[43]  van der Linden,et al.  Constrained adaptive testing with shadow tests , 2000 .

[44]  Philip H. Dubois,et al.  Correlational Methods in Research on Human Learning , 1962 .

[45]  Hua-Hua Chang,et al.  A Global Information Approach to Computerized Adaptive Testing , 1996 .

[46]  Susan E. Embretson,et al.  Measurement Model for Linking Individual Learning to Processes and Knowledge: Application to Mathematical Reasoning , 1995 .

[47]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[48]  L. Cronbach,et al.  How we should measure "change": Or should we? , 1970 .