Preliminary Findings on the Computer-Administered Multiple-Choice Online Causal Comprehension Assessment, a Diagnostic Reading Comprehension Test

The computer-administered Multiple-Choice Online Causal Comprehension Assessment (MOCCA) for Grades 3 to 5 has an innovative, 40-item multiple-choice structure in which each distractor corresponds to a comprehension process upon which poor comprehenders have been shown to rely. This structure requires revised thinking about measurement issues (e.g., reliability and interpretation of incorrect responses for diagnostic purposes). Using data from a pilot study, the article presents descriptive statistics on correct responses, incorrect responses, and comprehension rate. It also presents reliability data for correct responses and incorrect responses as well as construct validity data on correct responses. Implications for diagnosis and remediation of poor inferential comprehension are discussed.

[1]  M. Habib.  Assessment of Reading Comprehension , 2016 .

[2]  Anna Brown.  Item Response Models for Forced-Choice Questionnaires: A Common Framework , 2014, Psychometrika.

[3]  C. Skinner,et al.  Accurate reading comprehension rate as an indicator of broad reading in students in first, second, and third grades. , 2015, Journal of school psychology.

[4]  Pamela E. Guess,et al.  THE UTILITY OF MAZE ACCURATE RESPONSE RATE IN ASSESSING READING COMPREHENSION IN UPPER ELEMENTARY AND MIDDLE SCHOOL STUDENTS , 2014 .

[5]  Laine Bradshaw,et al.  Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions , 2014, Psychometrika.

[6]  Kristen L. McMaster,et al.  Development of a new reading comprehension assessment: Identifying comprehension differences among readers , 2014 .

[7]  Kristen L. McMaster,et al.  Making Connections: Linking Cognitive Psychology and Intervention Research to Improve Comprehension of Struggling Readers , 2014 .

[8]  Mary Jane White,et al.  Making the right connections: Differential effects of reading intervention for subgroups of comprehenders , 2012 .

[9]  Charles Hulme,et al.  Children's Reading Comprehension Difficulties , 2011 .

[10]  Cari F. Herrmann-Abell,et al.  Using Distractor-Driven Standards-Based Multiple-Choice Assessments and Rasch Modeling to Investigate Hierarchies of Chemistry Misconceptions and Detect Structural Problems with Individual Items. , 2011 .

[11]  Jaime B. Henning,et al.  READING ASSESSMENT METHODS FOR MIDDLE-SCHOOL STUDENTS: AN INVESTIGATION OF READING COMPREHENSION RATE AND MAZE ACCURATE RESPONSE RATE , 2011 .

[12]  Kate Nation,et al.  Suppressing Irrelevant Information from Working Memory: Evidence for Domain-Specific Deficits in Poor Comprehenders. , 2010 .

[13]  Daniel M. Bolt,et al.  On the Use of Factor-Analytic Multinomial Logit Item Response Models to Account for Individual Differences in Response Style , 2010 .

[14]  Renee O. Hawkins,et al.  The validity of reading comprehension rate: Reading speed, comprehension, and comprehension rates , 2009 .

[15]  Beth Chance,et al.  ASSESSING STUDENTS’ CONCEPTUAL UNDERSTANDING AFTER A FIRST COURSE IN STATISTICS , 2007 .

[16]  S. Samuels.  The DIBELS tests: Is speed of barking at print what we mean by reading fluency? , 2007 .

[17]  David N. Rapp,et al.  Higher-Order Comprehension Processes in Struggling Readers: A Perspective for Research and Intervention , 2007 .

[18]  Charles A. Perfetti,et al.  Reading Ability: Lexical Quality to Comprehension , 2007 .

[19]  Renee O. Hawkins,et al.  A preliminary investigation of the concurrent validity of reading comprehension rate: A direct, dynamic measure of reading comprehension , 2007 .

[20]  Jane Oakhill,et al.  Profiles of children with specific reading comprehension difficulties. , 2006, The British journal of educational psychology.

[21]  S. Samuels,et al.  Role of Automaticity in Metacognition and Literacy Instruction , 2006 .

[22]  Joanna P. Williams.  Stories, Studies, and Suggestions About Reading , 2006 .

[23]  Michael C. Rodriguez.  Three Options Are Optimal for Multiple‐Choice Items: A Meta‐Analysis of 80 Years of Research , 2005 .

[24]  P. David Pearson,et al.  The Assessment of Reading Comprehension: A Review of Practices—Past, Present, and Future , 2005 .

[25]  Hugh W. Catts,et al.  Developmental Changes in Reading and Reading Disabilities , 2005 .

[26]  Hugh W. Catts,et al.  The Connections between Language and Reading Disabilities , 2005 .

[27]  B. Wise,et al.  The Assessment of Reading Comprehension: Considerations and Cautions , 2005 .

[28]  Jeanne Sternlicht Chall,et al.  Poor Children's Fourth-Grade Slump. , 2003 .

[29]  C. Skinner,et al.  Advances in Curriculum-Based Measurement: Alternative Rate Measures for Assessing Reading Skills in Pre- and Advanced Readers , 2002 .

[30]  Catherine Snow,et al.  Reading for Understanding: Toward an R&D Program in Reading Comprehension , 2002 .

[31]  Jane Oakhill,et al.  Inference making ability and its relation to comprehension failure in young children , 1999 .

[32]  P. David Pearson,et al.  Two Steps Forward, Three Steps Back: The Stormy History of Reading Comprehension Assessment , 1998 .

[33]  Paul van den Broek,et al.  AUTOMATICITY AND INFERENCE GENERATION DURING READING COMPREHENSION , 1997 .

[34]  S. Jay Samuels,et al.  THE IMPORTANCE OF AUTOMATICITY FOR DEVELOPING EXPERTISE IN READING , 1997 .

[35]  Gordon D. Logan,et al.  Automaticity and Reading: Perspectives from the Instance Theory of Automatization. , 1997 .

[36]  Raymond J. Adams,et al.  The Multidimensional Random Coefficients Multinomial Logit Model , 1997 .

[37]  Paul van den Broek,et al.  Discovering the cement of the universe: The development of event comprehension from childhood to adulthood. , 1997 .

[38]  Sheila W. Valencia,et al.  Issues in Literacy Assessment: Facing the Realities of Internal and External Assessment , 1994 .

[39]  T. Trabasso,et al.  Constructing inferences during narrative text comprehension. , 1994, Psychological review.

[40]  K. A. Ericsson,et al.  Protocol analysis: Verbal reports as data, Rev. ed. , 1993 .

[41]  D. Hestenes,et al.  Force concept inventory , 1992 .

[42]  P. Broek.  The causal inference maker: Towards a process model of inference generation in text comprehension. , 1990 .

[43]  S. Deno,et al.  Curriculum-Based Measurement: The Emerging Alternative , 1985, Exceptional children.

[44]  T. Trabasso,et al.  Causal thinking and the representation of narrative events , 1985 .

[45]  A. Graesser,et al.  The Generation of Knowledge - Based Inferences During Narrative Comprehension , 1985 .

[46]  F. Samejima.  A New Family of Models for the Multiple-Choice Item. , 1979 .

[47]  Walter Kintsch,et al.  Toward a model of text comprehension and production. , 1978 .

[48]  Robert Schreiner,et al.  Assessing Reading Comprehension. , 1977 .

[49]  Charles A. Perfetti,et al.  Coding and Comprehension in Skilled Reading and Implications for Reading Instruction. , 1977 .

[50]  Alan S. Brown,et al.  Information Processing and Cognition: The Loyola Symposium , 1976 .

[51]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[52]  M. Posner,et al.  Attention and cognitive control. , 1975 .

[53]  S. Jay Samuels,et al.  Toward a theory of automatic information processing in reading , 1974 .

[54]  R. Darrell Bock,et al.  Estimating item parameters and latent ability when responses are scored in two or more nominal categories , 1972 .

[55]  R. M. Thorndike.  Measurement and Evaluation in Psychology and Education , 1969 .