Investigating the Score Dependability of an English Reading Comprehension Test

This study examined the score dependability of an English reading comprehension test using multivariate generalizability theory (G-theory). There were two main goals of the study: (1) investigate the score dependability of the reading test and (2) to examine the text effect in the reading test. Participants were 298 adult students to an English program and they took a reading test with four text and 30 multiple-choice items. The test was designed to measure two constructs of (1) reading for literal meaning and (2) reading for implied meaning. Multivariate G-theory analysis results indicated that the test had high score dependability, thereby accurately estimated the examinee’s universe-score. In addition, the two reading constructs were found to be highly correlated, yet separable. Moreover, the reading texts had a sizable effect on test scores. Findings indicated that increasing the number of texts and items would increase score dependability. However, increasing only the number of items deemed most reasonable.