Capturing, Codifying and Scoring Complex Data for Innovative, Computer-Based Items.

The Microsoft Certification Program (MCP) includes many new computer-based item types, based on complex cases involving the Windows 2000 (registered) operating system. This Innovative Item Technology (IIT) has presented challenges beyond traditional psychometric considerations such as capturing and storing the relevant response data from examinees, codifying it, and scoring it. This paper presents an integrated, empirically based data-systems approach to processing complex scoring rules and examinee response data for IIT cases and items, highlighting data management considerations, use of various psychometric analyses to improve the coding and scoring rules, and some of the challenges of scaling the data, using a partial credit item response theory model. Empirical examples from the Microsoft Innovative Item Types project are used to illustrate practical problems and solutions. (Author/SLD) Reproductions supplied by EDRS are the best that can be made from the original document. 1 PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement jUCATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received from the person or organization originating it. 0 Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. Capturing, Codifying and Scoring Complex Data for Innovative, Computer-based Items1 Richard M. Luecht University of North Carolina at Greensboro