Are Tests Comprising Both Multiple‐Choice and Free‐Response Items Necessarily Less Unidimensional Than Multiple‐Choice Tests?An Analysis of Two Tests

We consider the relationship between the multiple-choice and free-response sections on the Computer Science and Chemistry tests of the College Board's Advanced Placement program. Restricted factor analysis shows that the free-response sections measure the same underlying proficiency as the multiple-choice sections for the most part. However, there is also a significant, if relatively small, amount of local dependence among the free-response items that produces a small degree of multidimensionauty for each test

[1]  R. Cattell,et al.  Validation and intensification of the Sixteen Personality Factor Questionnaire. , 1956, Journal of clinical psychology.

[2]  John Schmid,et al.  The development of hierarchical factor solutions , 1957 .

[3]  R. Cattell.  Radial parcel factoring-vs-item factoring in defining personality structure in questionnaires: Theory and experimental checks , 1974 .

[4]  Raymond B. Cattell,et al.  The Radial Parcel Double Factoring Design: A Solution to the Item-Vs-Parcel Controversy , 1975 .

[5]  Michael W. Browne,et al.  Topics in Applied Multivariate Analysis: COVARIANCE STRUCTURES , 1982 .

[6]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[7]  N. Dorans,et al.  THE INTERNAL CONSTRUCT VALIDITY OF THE SAT1 , 1987 .

[8]  Howard Wainer,et al.  Item Clusters and Computerized Adaptive Testing: A Case for Testlets , 1987 .

[9]  Norman Cliff,et al.  The eigenvalues-greater-than-one rule and the reliability of components. , 1988 .

[10]  E. Muraki,et al.  Full-Information Item Factor Analysis , 1988 .

[11]  Henry Braun,et al.  The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items , 1990 .

[12]  William Stout,et al.  A New Item Response Theory Modeling Approach with Applications to Unidimensionality Assessment and Ability Estimation , 1990 .

[13]  Randy Elliot Bennett,et al.  Equivalence of Free-Response and Multiple-Choice Items , 1991 .

[14]  Wendy M. Yen,et al.  Scaling Performance Assessments: Strategies for Managing Local Item Dependence , 1993 .