FOCUS ARTICLE: The Foundations of Assessment

This article presents major messages from the National Research Council report, Knowing What Students Know: The Science and Design of Educational Assessment (2001). The committee issuing this report was charged with synthesizing advances in the cognitive sciences and measurement, and exploring their implications for improving educational assessment. The article opens with a vision for the future of educational assessment that represents a significant departure from the types of assessments typically available today, and from the ways in which such assessments are most commonly used. This vision is driven by an interpretation of what is both necessary and possible for educational assessment to positively impact student achievement. The argument is made that realizing this vision requires a fundamental rethinking of the foundations and principles guiding assessment design and use. These foundations and principles and their implications are then summarized in the remainder of the article. The argument is made that every assessment, regardless of its purpose, rests on three pillars: (1) a model of how students represent knowledge and develop competence in the subject domain, (2) tasks or situations that allow one to observe students' performance, and (3) interpretation methods for drawing inferences from the performance evidence collected. These three elements-cognition, observation, and interpretation-must be explicitly connected and designed as a coordinated whole. Section II summarizes research and theory on thinking and learning which should serve as the source of the cognition element of the assessment triangle. This large body of research suggests aspects of student achievement that one would want to make inferences about, and the types of observations, or tasks, that will provide evidence to support those inferences. Also described are significant advances in methods of educational measurement that make new approaches to assessment feasible. The argument is presented that measurement models, which are statistical examples of the interpretation element of the assessment triangle, are cuuently available to support the kinds of inferences about student achievement that cognitive science suggests are important to pursue. Section III describes how the contemporary understanding of cognition and methods of measurement jointly provide a set of principles and methods for guiding the processes of assessment design and use. This section explores how the scientific foundations presented in Section II play out in the design of real assessment situations ranging from classroom to large-scale testing contexts. It also considers the role of technology in enhancing assessment design and use. Section IV presents a discussion of the research, development, policy, and practice issues that must be addressed for the field of assessment to move forward and achieve the vision described in Section I.

[1]  Raymond J. Adams,et al.  Multilevel Item Response Models: An Approach to Errors in Variables Regression , 1997 .

[2]  D. Sadler Formative assessment and the design of instructional systems , 1989 .

[3]  J. Frederiksen,et al.  Inquiry, Modeling, and Metacognition: Making Science Accessible to All Students , 1998 .

[4]  S. Lane,et al.  GENDER-RELATED DIFFERENTIAL ITEM FUNCTIONING ON A MIDDLE-SCHOOL MATHEMATICS PERFORMANCE ASSESSMENT , 2005 .

[5]  R. Glaser Expert knowledge and processes of thinking , 1992 .

[6]  Susy Macqueen,et al.  Validity , 1973, Just Algorithms.

[7]  Dorothy L. Gabel,et al.  Handbook of Research on Science Teaching and Learning Project. , 1993 .

[8]  Robert J. Mislevy,et al.  Test Theory for A New Generation of Tests , 1994 .

[9]  Robert A. Bjork,et al.  On the puzzling relationship between environmental context and human memory. , 1989 .

[10]  Robert J. Mislevy,et al.  FOUNDATIONS OF A NEW TEST THEORY , 1982 .

[11]  W. James Popham,et al.  Modern Educational Measurement: Practical Guidelines for Educational Leaders , 1999 .

[12]  W. Brewer,et al.  Mental models of the earth: A study of conceptual change in childhood , 1992, Cognitive Psychology.

[13]  Charting Student Progress , 1989 .

[14]  J. Lagowski National Science Education Standards , 1995 .

[15]  James W. Pellegrino,et al.  Chapter 9 : Addressing the “Two Disciplines” Problem: Linking Theories of Cognition and Learning With Assessment and Instructional Practice , 1999 .

[16]  R. Linn Assessments and Accountability , 2000 .

[17]  D. Hunt Cognition and Learning , 1989 .

[18]  Susan E. Embretson,et al.  Multicomponent Response Models , 1997 .

[19]  Francis Tuerlinckx,et al.  A Hierarchical IRT Model for Criterion-Referenced Measurement , 2000 .

[20]  B. Rogoff Apprenticeship in Thinking: Cognitive Development in Social Context , 1990 .

[21]  L. Shepard The Role of Assessment in a Learning Culture , 2000 .

[22]  K. Ercikan,et al.  Analysis of Differential Item Functioning in the NAEP History Assessment , 1988 .

[23]  Robbie Case,et al.  The Mind's Staircase: Exploring the Conceptual Underpinnings of Children's Thought and Knowledge , 1991 .

[24]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[25]  C. Steele A threat in the air. How stereotypes shape intellectual identity and performance. , 1997, The American psychologist.

[26]  Ron Stevens,et al.  The Use of Artificial Neural Nets (ANN) to Help Evaluate Student Problem Solving Strategies , 2000 .

[27]  L. Quéré Cognition in Practice , 1996 .

[28]  Mark R. Wilson,et al.  Running Head: Measurement at the Knowledge Level. A Theory of the Measurement of Knowledge Content, Access, and Learning. , 1996 .

[29]  Kikumi K. Tatsuoka,et al.  Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. , 1995 .

[30]  R. Brennan Elements of generalizability theory , 1983 .

[31]  D. Geary,et al.  Reflections of evolution and culture in children's cognition. Implications for mathematical development and instruction. , 1995, The American psychologist.

[32]  M. McCloskey,et al.  The development of beliefs about falling objects , 1985, Perception & psychophysics.

[33]  J. Shea National Science Education Standards , 1995 .

[34]  L. Fuchs,et al.  Effects of Systematic Formative Evaluation: A Meta-Analysis , 1986, Exceptional children.

[35]  John D. Bransford,et al.  The Jasper Project: Lessons in Curriculum, Instruction, Assessment, and Professional Development , 1997 .

[36]  Robert Glaser,et al.  Investigating the Cognitive Complexity of Science Assessments , 1998 .

[37]  최영한,et al.  미국 NCTM의 Principles and Standards for School Mathematics에 나타난 수학과 교수,학습의 이론 , 2002 .

[38]  Howard Gardner,et al.  To Use Their Minds Well: Investigating New Forms of Student Assessment , 1991 .

[39]  K. Ercikan,et al.  Translation effects in international assessments , 1998 .

[40]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .

[41]  Gerry Stahl,et al.  Developing Summarization Skills through the Use of LSA-Based Feedback , 2000, Interact. Learn. Environ..

[42]  Jay P. Heubert,et al.  High stakes : testing for tracking, promotion, and graduation , 1999 .

[43]  Jamal Abedi,et al.  NAEP Math Performance and Test Accommodations: Interactions with Student Language Background. CSE Technical Report. , 2001 .

[44]  Raymond J. Adams,et al.  The Multidimensional Random Coefficients Multinomial Logit Model , 1997 .

[45]  R. Gelman,et al.  Preschooler's Ability to Decide Whether a Photographed Unfamiliar Object Can Move Itself. , 1988 .

[46]  Robert S. Siegler,et al.  Rightstart: Providing the central conceptual prerequisites for first formal learning of arithmetic to students at risk for school failure. , 1994 .

[47]  K. Crowley,et al.  The microgenetic method. A direct means for studying cognitive development. , 1991, The American psychologist.

[48]  S. Heath Ways with Words: Language, Life and Work in Communities and Classrooms , 1983 .

[49]  Daniel Koretz,et al.  The Validity of Gains in Scores on the Kentucky Instructional Results Information System (KIRIS). , 1998 .

[50]  Michelene T. H. Chi,et al.  Expertise in Problem Solving. , 1981 .

[51]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[52]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[53]  B. Junker Some statistical models and computational methods that may be useful for cognitively-relevant assessment , 1999 .

[54]  Robert Glaser,et al.  Instructional technology and the measurement of learing outcomes: Some questions. , 1963 .

[55]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[56]  R. Mislevy,et al.  Marginal maximum likelihood estimation for a psychometric model of discontinuous development , 1996 .

[57]  James W. Pellegrino,et al.  Grading the Nation's Report Card: Evaluating NAEP and Transforming the Assessment of Educational Progress. , 1999 .

[58]  John R. Anderson,et al.  Cognitive Modeling and Intelligent Tutoring , 1990, Artif. Intell..

[59]  E. Michael Nussbaum,et al.  Interview Procedures for Validating Science Assessments , 1997 .

[60]  Robert Glaser,et al.  Assessment in Transition: Monitoring the Nation's Educational Progress. , 1997 .

[61]  Paul Cobb,et al.  Assessment of a problem-centered second-grade mathematics project. , 1991 .

[62]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[63]  C. Gallistel,et al.  The Child's Understanding of Number , 1979 .

[64]  Andy MacLeod,et al.  Making space for IT , 2005 .

[65]  R. Driver,et al.  Making Sense of Secondary Science: Research into children’s ideas , 1993 .

[66]  Mark Wilson,et al.  From Principles to Practice: An Embedded Assessment System , 2000 .

[67]  Hyman Bass,et al.  Chapter VII: Making Believe: The Collective Construction of Public Mathematical Knowledge in the Elementary Classroom , 2000, Teachers College Record: The Voice of Scholarship in Education.

[68]  Edward H. Haertel Continuous and discrete latent structure models for item response data , 1990 .

[69]  Robbie Case,et al.  The Role of Central Conceptual Structures in the Development of Children's Thought , 1995 .

[70]  A. Su,et al.  The National Council of Teachers of Mathematics , 1932, The Mathematical Gazette.

[71]  Robert S. Siegler,et al.  Children's thinking, 2nd ed. , 1991 .

[72]  G. Cizek Pockets of Resistance in the Assessment Revolution , 2005 .

[73]  M. Linn,et al.  Beyond Fourth-Grade Science: Why Do U.S. and Japanese Students Diverge? , 2000 .

[74]  D. Klahr,et al.  Knowing about guessing and guessing about knowing: preschoolers' understanding of indeterminacy. , 1996, Child development.

[75]  R. Stiggins Student-centered classroom assessment , 1994 .

[76]  Paul Cobb,et al.  Participating in Classroom Mathematical Practices , 2001 .

[77]  John B. Willett,et al.  Using covariance structure analysis to detect correlates and predictors of individual change over time , 1994 .

[78]  Ning Wang,et al.  Validating the cognitive complexity and content quality of a mathematics performance assessment , 1994 .

[79]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[80]  Robert J. Mislevy,et al.  TEST THEORY RECONCEIVED , 1994 .

[81]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .