Use of Response Process Data to Inform Group Comparisons and Fairness Research

ABSTRACT Comparing group is one of the key uses of large-scale assessment results, which are used to gain insights to inform policy and practice and to examine the comparability of scores and score meaning. Such comparisons typically focus on examinees’ final answers and responses to test questions, ignoring response process differences groups may engage in. This paper discusses and demonstrates the use of response process data in enhancing group comparison and fairness research methodologies. We propose two statistical approaches for identifying differential response processes which extend the differential item functioning (DIF) detection methods and demonstrate the complementary use of process data in comparing groups in two case studies Our findings demonstrate the use of response process data in gaining insights about students’ test-taking behaviors from different populations that go beyond what may be identified using response data only.