Computerized Adaptive and Fixed‐Item Testing of Music Listening Skill: A Comparison of Efficiency, Precision, and Concurrent Validity

We evaluated the efficiency, precision, and concurrent validity of results obtained from adaptive and fired-item music listening tests in three studies: (a) a computer simulation study in which each of 2,200 simulees completed a computerized adaptive tonal memory test, a computerized fired-item tonal memory test constructed from items in the adaptive test pool and two standardized group-administered tonal memory tests; (b) a live testing study in which each of 204 examinees took the computerized adaptive test and the standardized tests; and (c) a live testing study in which randomly equivalent groups took either the computerized adaptive test (n = 86) or the computerized fired-item test (n = 86). The adaptive music test required 50% to 93% fewer items to match the reliability and concurrent validity of the fired-item tests, and it yielded higher levels of reliability and concurrent validity than the fired-item tests when test length was held constant. These findings suggest that computerized adaptive tests, which typically have been limited to visually produced items, may also be well suited for measuring skills that require aurally produced items.

[1]  James B. Olsen,et al.  Comparisons of Paper-Administered, Computer-Administered and Computerized Adaptive Achievement Tests , 1989 .

[2]  Frederic M. Lord,et al.  Estimation of latent ability and item parameters when there are omitted responses , 1974 .

[3]  Roberto de-la-Torre,et al.  The Development and Evaluation of a Computerized Adaptive Testing System. , 1991 .

[4]  J. Mcbride,et al.  Reliability and Validity of Adaptive Ability Tests in a Military Setting , 1983 .

[5]  David J. Weiss,et al.  An Adaptive Testing Strategy for Achievement Test Batteries. , 1977 .

[6]  David J. Weiss,et al.  Predictive validity of conventional and adaptive tests in an Air Force training environment.: (406052004-001) , 1982 .

[7]  John L. Horn,et al.  Human Abilities Revealed Through Auditory Tests , 1980 .

[8]  David J. Weiss,et al.  Improving Measurement Quality and Efficiency with Adaptive Testing , 1982 .

[9]  R. Hambleton.  Principles and selected applications of item response theory. , 1989 .

[10]  John E. Stecklein,et al.  Aliferis-Stecklein Music Achievement Test, College Midpoint Level , 1963 .

[11]  Walter P. Vispoel,et al.  The Development and Evaluation of a Computerized Adaptive Test of Tonal Memory , 1993 .

[12]  J. D. Boyle,et al.  Iowa Tests of Music Literacy , 1973 .

[13]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[14]  Anne L. Harvey,et al.  The Equivalence of Scores from Automated and Conventional Educational and Psychological Tests: A Review of the Literature. College Board Report No. 88-8. , 1988 .

[15]  W. M. Yen.  Using Simulation Results to Choose a Latent Trait Model , 1981 .

[16]  F. Lord.  A Broad-Range Tailored Test of Verbal Ability , 1975 .

[17]  Betty A. Bergstrom,et al.  The Effect of Review on Student Ability and Test Efficiency for Computerized Adaptive Tests , 1992 .

[18]  Vern W. Urry,et al.  TAILORED TESTING: A SUCCESSFUL APPLICATION OF LATENT TRAIT THEORY* , 1977 .

[19]  Walter P. Vispoel,et al.  Computerized Adaptive Testing of Music-Related Skills. , 1992 .

[20]  David J. Weiss,et al.  APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS , 1984 .

[21]  Walter P. Vispoel.  How Review Options and Administration Modes Influence Scores on Computerized Vocabulary Tests. , 1992 .

[22]  Howard Wainer,et al.  CATS, TESTLETS, AND TEST CONSTRUCTION: A RATIONALE FOR PUTTING TEST DEVELOPERS BACK INTO CAT , 1986 .

[23]  H. Wing,et al.  Tests of musical ability and appreciation , 1948 .

[24]  H. Gardner,et al.  Frames of Mind: The Theory of Multiple Intelligences , 1983 .

[25]  Rudolf E. Radocy,et al.  Measurement and Evaluation of Musical Experiences , 1987 .

[26]  Dennis R. Bowers.  Computer-based adaptive testing in music research and instruction. , 1991 .

[27]  D. Weiss.  Adaptive testing by computer. , 1985, Journal of consulting and clinical psychology.

[28]  David J. Weiss,et al.  Book Review : New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing David J. Weiss (Ed.) New York: Academic Press, 1983, 345 pp., $35.00 , 1984 .

[29]  Kathleen E. Moreno,et al.  Relationship Between Corresponding Armed Services Vocational Aptitude Battery (ASVAB) and Computerized Adaptive Testing (CAT) Subtests , 1984 .

[30]  Walter P. Vispoel,et al.  Improving the measurement of tonal memory with computerized adaptive tests. , 1992 .

[31]  J. B. Olsen,et al.  THE FOUR GENERATIONS OF COMPUTERIZED EDUCATIONAL MEASUREMENT , 1988 .

[32]  R. D. Bock,et al.  Adaptive EAP Estimation of Ability in a Microcomputer Environment , 1982 .

[33]  John Davies,et al.  New Tests of Musical Aptitude. , 1971 .

[34]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .