DIFFERENCES IN INSTRUCTIONAL SENSITIVITY BETWEEN ITEM FORMATS AND BETWEEN ACHIEVEMENT TEST ITEMS

Building achievement tests which are sensitive to the instructional effects of school programs concerns both practitioners and researchers in education. To produce such tests, empirical procedures to guide item selection are needed. In this paper, an operational framework and a set of empirical procedures for this task are presented. Within this framework, item sensitivity is linked to instructional implementation. A simple components of variance model has been used to provide actual estimates of instructional sensitivity. These procedures are illustrated using data from a comparative study of alternative item formats for a criterion-referenced test. Even when items were closely matched to instructional content specifications, important differences in instructional sensitivity emerged. These differences were found between the same items presented in different formats as well as between different items presented within the same format. Implications of these results for developing criterion-referenced achievement tests are discussed.

[1]  A. W. Heim,et al.  AN EXPERIMENT ON MULTIPLE-CHOICE VERSUS OPEN-ENDED ANSWERING IN A VOCABULARY TEST , 1967 .

[2]  Ross E. Traub,et al.  On the Equivalence of Constructed- Response and Multiple-Choice Tests , 1977 .

[3]  Peter W. Airasian,et al.  ASSESSING SCHOOL AND PROGRAM EFFECTIVENESS: ESTIMATING TEACHER LEVEL EFFECTS , 1978 .

[4]  W. James Popham,et al.  Criterion-Referenced Measurement , 1971 .

[5]  E. A. Haggard.  Intraclass correlation and the analysis of variance. , 1960 .

[6]  David M. Shoemaker,et al.  Toward a Framework for Achievement Testing , 1975 .

[7]  Peter W. Airasian,et al.  School effectiveness: A reassessment of the evidence , 1980 .

[8]  Paul Berman,et al.  Federal Programs Supporting Educational Change, Vol. VII: Factors Affecting Implementation and Continuation. , 1977 .

[9]  R. Ebel.  Content Standard Test Scores , 1962 .

[10]  Jane A. Stallings.  Allocated Academic Learning Time Revisited, or Beyond Time on Task , 1980 .

[11]  Gaea Leinhardt,et al.  Modeling and Measuring Educational Treatment in Evaluation , 1980 .

[12]  C. Bird,et al.  A comparison of two new-type questions: recall and recognition. , 1938 .

[13]  E. A. Haggard,et al.  Intraclass Correlation and the Analysis of Variance , 1958 .

[14]  Anthony J. Nitko,et al.  MEASUREMENT IN LEARNING AND INSTRUCTION. , 1970 .

[15]  Leon M. Lessinger,et al.  Accountability in education , 1971 .

[16]  Anthony J. Nitko.  Criterion-Referencing Schemes. , 1980 .

[17]  O. K. Buros.  Fifty Years in Testing: Some Reminiscences, Criticisms, and Suggestions , 1977 .

[18]  A. Lewy.  DISCRIMINATION AMONG INDIVIDUALS V. DISCRIMINATION AMONG GROUPS , 1973 .

[19]  Robert Glaser,et al.  Instructional technology and the measurement of learing outcomes: Some questions. , 1963 .

[20]  Richard F. Elmore,et al.  Social Program Implementation. , 1977 .

[21]  Ronald A. Berk,et al.  Criterion-Referenced Measurement: The State of the Art , 1980 .

[22]  B. Bloom,et al.  Evaluation to Improve Learning , 1981 .

[23]  Ronald P. Carver,et al.  Two Dimensions of Tests: Psychometric and Edumetric. , 1974 .

[24]  R. D. Bock,et al.  Multivariate Statistical Methods in Behavioral Research , 1978 .

[25]  Quasi-Experimentation in Educational Settings , 1967, The School Review.

[26]  T. R. Husek,et al.  IMPLICATIONS OF CRITERION‐REFERENCED MEASUREMENT1,2 , 1969 .

[27]  H. Walberg.  Improving the Productivity of America's Schools. , 1984 .

[28]  F. Graybill.  An introduction to linear statistical models , 1961 .

[29]  Paul E. Resta,et al.  Installation Requirements for the SWRL First-Year Communication Skills Program: Evaluation Data, 1968-69. , 1971 .

[30]  Ronald K. Hambleton,et al.  DETERMINING THE LENGTHS FOR CRITERION-REFERENCED TESTS , 1983 .

[31]  Peter W. Airasian,et al.  LINKING TESTING AND INSTRUCTION: POLICY ISSUES , 1983 .

[32]  Benjamin S. Bloom,et al.  All Our Children Learning , 1982 .

[33]  R. D. Bock,et al.  Quasi-Experimentation in Educational Settings: Comment , 1967, The School Review.

[34]  R. Elmore,et al.  Organizational models of social program implementation. , 1978, Public policy.