Measuring Reading Comprehension with the Lexile Framework.

This paper shows how the concept of general objectivity can be used to improve behavioral science measurement, particularly as it applies to the Lexile Framework, a tool for objectively measuring reading comprehension. It begins with a dialogue between a physicist and a psychometrician that details some of the differences between physical science and behavioral science measurement. Building on these distinctions, a definition of measurement is offered that describes what goes on in the physical sciences and represents an attainable ideal of what should go on in the behavioral sciences. This definition of measurement is formalized in an equation that turns out to be the Rasch model, with the important difference that indicant calibrations are obtained via theory, not data. Through the use of theory-based calibrations, a generally objective estimation of the measure parameter in the Rasch model is achieved. The paper then examines the differences between local objectivity obtained with the Rasch model and general objectivity obtained with a theory-enhanced version of that model. Next, it reports on a 10-year study of reading comprehension measurement that implemented the concept of general objectivity through the development of the Lexile Framework. Finally, it summarizes several of the benefits of objective measurement and general objectivity as they might be realized in the measurement of constructs other than reading comprehension. Contains 52 references, and 4 tables and a figure of data. (EF) Reproductions supplied by EDRS are the best that can be made from the original document. Measuring Reading Comprehension with the Lexile Framework A. Jackson Stenner Meta Metrics, Inc. Paper Presented at the Fourth North American Conference on Adolescent/Adult Literacy Washington, D.C. February 1996 U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) flifThis document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. Meta Metrics, Inc. 2327 Eng lert Drive Suite 300 Durham NC 27713 Telephone (919) 547-3400; Fax (919) 547-3401

[1]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[2]  R. Luce,et al.  Simultaneous conjoint measurement: A new type of fundamental measurement , 1964 .

[3]  V. Mann,et al.  Children's Memory for Recurring Linguistic and Nonlinguistic Material in Relation to Reading Ability , 1982, Cortex.

[4]  B. Wright,et al.  Best test design , 1979 .

[5]  A. Stenner.  Most Comprehension Tests Do Measure Reading Comprehension: A Response to McLean and Goldstein. , 1988 .

[6]  A. Jackson Stenner,et al.  TOWARD A THEORY OF CONSTRUCT DEFINITION , 1983 .

[7]  Benjamin D. Wright,et al.  SAMPLE-FREE TEST CALIBRATION AND PERSON MEASUREMENT. PAPER PRESENTED AT THE NATIONAL SEMINAR ON ADULT EDUCATION RESEARCH (CHICAGO, FEBRUARY 11-13, 1968). , 1967 .

[8]  C. Perfetti,et al.  Linguistic complexity and text comprehension : readability issues reconsidered , 1989 .

[9]  Alan D. Baddeley,et al.  Verbal Reasoning and Working Memory , 1976 .

[10]  J W Hedge,et al.  Personnel selection. , 1997, Annual review of psychology.

[11]  John B. Carroll,et al.  The American Heritage Word Frequency Book , 1971 .

[12]  James V. Mitchell.  The ninth mental measurements yearbook , 1985 .

[13]  Richard W. Woodcock,et al.  Woodcock reading mastery tests , 1987 .

[14]  Larry A. Harris,et al.  Woodcock reading mastery tests: Woodcock, R. Circle Pines, Minn.: American Guidance Service, 1974. Form A or B: $18 , 1976 .

[15]  John R. Bormuth,et al.  READABILITY--A NEW APPROACH. , 1966 .

[16]  G. Rasch,et al.  A MATHEMATICAL THEORY OF OBJECTIVITY AND ITS CONSEQUENCES FOR MODEL CONSTRUCTION , 1968 .

[17]  Brenda Rogers,et al.  The Key Math Diagnostic Arithmetic Test , 1981 .

[18]  A. Jackson Stenner,et al.  Testing Construct Theories , 1982 .

[19]  Donald Shankweilert,et al.  SYNTACTIC COMPLEXITY AND READING ACQUISITION , 1986 .

[20]  G. Miller,et al.  How children learn words. , 1987, Scientific American.

[21]  Marion Perlmutter,et al.  Children's memory , 1980 .

[22]  Andrew B. Hahn.  Reaching Out to America's Dropouts: What to Do?. , 1987 .

[23]  F. Markwardt.  Peabody Individual Achievement Test-Revised , 1989 .

[24]  Robert N. Kantor,et al.  On the Failure of Readability Formulas to Define Readable Texts: A Case Study from Adaptations. , 1982 .

[25]  L. Thurstone.  Attitudes Can Be Measured , 1928, American Journal of Sociology.

[26]  David A. Squires,et al.  Effective Schools and Classrooms: A Research-Based Perspective , 1983 .

[27]  Stephen Crain,et al.  Language mechanisms and reading disorder: A modular approach , 1986, Cognition.

[28]  Richard C. Anderson.  Becoming a Nation of Readers: The Report of the Commission on Reading , 1985 .

[29]  Ronald P. Carver,et al.  Measuring the Primary Effect of Reading: Reading-Storage Technique, Understanding Judgments, and Cloze* , 1974 .

[30]  Richard C. Anderson,et al.  Conceptual and empirical bases of readability formulas , 1986 .

[31]  George R. Klare,et al.  The measurement of readability , 1963 .

[32]  J. Chall,et al.  Readability revisited : the new Dale-Chall readability formula , 1995 .

[33]  L. L. Thurstone,et al.  A method of scaling psychological and educational tests. , 1925 .

[34]  E. R. Paul.  The Beginning Years , 1993 .

[35]  W. John Crawford.  Error Rates and Question Difficulty Related to Elementary Children's Learning. Report No. 75-8. , 1975 .