Techniques and Procedures for Assessing Cognitive Skills

The intent of the article is to survey procedures that could be used to assess progress in instructional programs designed to enhance cognitive skills. The organizational framework is provided by J. R. Anderson’s (1982) theory of cognitive skill development and by Glaser, Lesgold, and Lajoie’s (1985) categorization of dimensions of cognitive skills. After describing Anderson’s theory, the article discusses the following types of measures of cognitive skills: (a) measures of knowledge acquisition, organization, and structure; (b) measures of depth of problem representation; (c) measures of mental models; (d) measures of metacognitive skills; (e) measures of the automaticity of performance; and (f) measures of efficiency of procedures. Each of the sections describing measurement procedures is followed by a discussion of the strengths and weaknesses of the procedures. The article closes with a general discussion of techniques for measuring cognitive skills.

[1]  P. Johnson.  On the communication of concepts in science. , 1969 .

[2]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[3]  R. B. Allen,et al.  Computer text editing , 1984 .

[4]  John R. Anderson.  Acquisition of cognitive skill. , 1982 .

[5]  S. Messick.  Test validity and the ethics of assessment. , 1980 .

[6]  A. Caramazza,et al.  Curvilinear motion in the absence of external forces: naive beliefs about the motion of objects. , 1980, Science.

[7]  Lola L. Lopes.  Model-based decision and inference in stud poker. , 1976 .

[8]  Keith E. Stanovich,et al.  Concepts in developmental theories of reading skill: Cognitive resources, automaticity, and modularity. , 1990 .

[9]  Richard J. Shavelson,et al.  An Exploratory Analysis of the Representation of a Mathematical Structure in Students’ Cognitive Structures1 , 1975 .

[10]  John R. Anderson.  The Adaptive Character of Thought , 1990 .

[11]  Marlene Scardamalia,et al.  Teachability of Reflective Processes in Written Composition , 1984, Cogn. Sci..

[12]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[13]  Allen Newell,et al.  GPS, a program that simulates human thought , 1995 .

[14]  Woodrow Barfield,et al.  Expert-novice differences for software: implications for problem-solving and knowledge acquisition , 1986 .

[15]  B. Adelson.  Problem solving and the development of abstract categories in programming languages , 1981, Memory & cognition.

[16]  Paul M. Fitts,et al.  Perceptual-Motor Skill Learning1 , 1964 .

[17]  James M. Royer,et al.  The Prediction of College Course Performance From Reading Comprehension Performance: Evidence for General and Specific Prediction Factors , 1990 .

[18]  John Sweller,et al.  Development of expertise in mathematical problem solving , 1983 .

[19]  James M. Royer,et al.  Using the Sentence Verification Technique To Assess the Comprehension of Technical Text as a Function of Subject Matter Expertise , 1984 .

[20]  Michelene T. H. Chi,et al.  Expertise in Problem Solving. , 1981 .

[21]  D. Gentner,et al.  Flowing waters or teeming crowds: Mental models of electricity , 1982 .

[22]  James M. Royer,et al.  Using entering reading comprehension performance as a predictor of performance in college classes , 1987 .

[23]  B. Adelson.  When Novices Surpass Experts: The Difficulty of a Task May Increase With Expertise , 1984 .

[24]  L. Resnick,et al.  Knowing, Learning, and Instruction , 2018 .

[25]  R. Sternberg,et al.  Complex Problem Solving : Principles and Mechanisms , 1992 .

[26]  R. Reynolds,et al.  Influence of questions on the allocation of attention during reading. , 1980, Journal of educational psychology.

[27]  Daniel G Bobrow,et al.  On data-limited and resource-limited processes , 1975, Cognitive Psychology.

[28]  D. Herrmann,et al.  Problem perception and knowledge structure in expert and novice mathematical problem solvers. , 1982 .

[29]  John T. Guthrie,et al.  Locating Information in Documents: Examination of a Cognitive Model. , 1988 .

[30]  Judith S Reitman,et al.  Organization revealed by recall orders and confirmed by pauses , 1980, Cognitive Psychology.

[31]  Richard C. Anderson,et al.  Educational psychology;: The science of instruction and learning , 1973 .

[32]  T. R. G. Green,et al.  ‘HARK-BACK’: A SIMPLE MEASURE OF SEARCH PATTERNS , 1976 .

[33]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[34]  R. Glaser.  Education and Thinking: The Role of Knowledge. , 1984 .

[35]  D E Egan,et al.  Chunking in recall of symbolic drawings , 1979, Memory & cognition.

[36]  Gavriel Salomon,et al.  Transcending the Qualitative-Quantitative Debate: The Analytic and Systemic Approaches to Educational Research , 1991 .

[37]  K A Ericcson,et al.  Acquisition of a memory skill. , 1980, Science.

[38]  Daniel Gopher,et al.  On the Economy of the Human Processing System: A Model of Multiple Capacity. , 1977 .

[39]  Mark Weiser,et al.  Programming Problem Representation in Novice and Expert Programmers , 1983, Int. J. Man Mach. Stud..

[40]  Allen Newell,et al.  Computer text-editing: An information-processing analysis of a routine cognitive skill , 1980, Cognitive Psychology.

[41]  F. Allard,et al.  Perception in Sport: Basketball , 1980 .

[42]  John R. Anderson.  Development of expertise , 1993 .

[43]  R. Shepard,et al.  Second-order isomorphism of internal representations: Shapes of states ☆ , 1970 .

[44]  Bruce K. Britton,et al.  Effects of prior knowledge on use of cognitive capacity in three complex cognitive tasks. , 1982 .

[45]  Iris Vessey,et al.  Expert-novice knowledge organization: an empirical investigation using computer program recall , 1988 .

[46]  James M. Royer.  The sentence verification technique: A new direction in the assessment of reading comprehension. , 1990 .

[47]  M. Fleury,et al.  Expertise Differences in Preparing to Return a Tennis Serve: A Visual Information Processing Approach , 1989 .

[48]  Clifford Konold,et al.  The episodic/semantic memory distinction as an heuristic in the study of instructional effects on cognitive structure , 1982 .

[49]  R. Glaser,et al.  Expertise in a complex skill: Diagnosing x-ray pictures. , 1988 .

[50]  S. Jay Samuels,et al.  Toward a theory of automatic information processing in reading , 1974 .

[51]  L. Baker.  Metacognition, comprehension monitoring, and the adult reader , 1989 .

[52]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[53]  D. Monty Neill,et al.  Standardized Testing: Harmful to Educational Health. , 1989 .

[54]  Yuichiro Anzai,et al.  Internal Models in Physics Problem Solving , 1984 .

[55]  C. Perfetti.  Verbal efficiency in reading ability. , 1988 .

[56]  J. Mestre,et al.  The relation between problem categorization and problem solving among experts and novices , 1989, Memory & cognition.

[57]  Helen E. Purkitt,et al.  An experimental study of cognitive processes and information in political problem solving , 1988 .

[58]  K. Holyoak.  Symbolic connectionism: Toward third-generation theories of expertise. , 1991 .

[59]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[60]  R. Shavelson.  Some Aspects of the Correspondence between Content Structure and Cognitive Structure in Physics Instruction. , 1972 .

[61]  John T. Guthrie,et al.  Roles of Document Structure, Cognitive Strategy, and Awareness in Searching for Information. , 1991 .

[62]  H. Simon,et al.  Perception in chess , 1973 .

[63]  Stephen J. Read,et al.  The effects of expertise on financial problem solving: Evidence for goal-directed, problem-solving scripts , 1990 .