Rapid Assessment of Learners’ Proficiency: A cognitive load approach

To tailor instruction to levels of learner proficiency in a domain in adaptive learning environments, rapid on‐line methods of assessing learner knowledge and skills are required. Cognitive studies indicate that major factors in expert performance are organised structures in the person’s knowledge base. Measuring different levels of acquisition of such knowledge structures should be a major purpose of cognitive diagnostic assessment. Traditional tests may not provide reliable evidence for diagnostic purposes and are not suitable for dynamic learning environments. This paper discusses an alternative diagnostic approach based on monitoring immediate traces of students’ knowledge structures in working memory. Results of an experimental study applying this method to assessment of user reading skills are described. The study demonstrated a sufficiently high level of correlation (.66) between obtained measures and traditional test scores, and a substantial increase in reliability, with testing time reduced by a factor of 3.8.

[1]  Slava Kalyuga,et al.  Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning , 2005 .

[2]  W. Kintsch.  How readers construct situation models for stories. The role of suntactic cues and causal inferences , 1992 .

[3]  James M. Royer,et al.  A Sentence Verification Technique for Measuring Reading Comprehension1 , 1979 .

[4]  Steven Schwartz.  Measuring Reading Competence: A Theoretical-Prescriptive Approach , 1985 .

[5]  Slava Kalyuga,et al.  Measuring Knowledge to Optimize Cognitive Load Factors During Instruction. , 2004 .

[6]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.

[7]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[8]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[9]  M. Just,et al.  Individual differences in syntactic processing: The role of working memory , 1991 .

[10]  J. Kimball.  Seven principles of surface structure parsing in natural language , 1973 .

[11]  Joseph P. Magliano,et al.  Assessing Reading Skill With a Think-Aloud Procedure and Latent Semantic Analysis , 2003 .

[12]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[13]  Marcel Adam Just,et al.  The Role of Working Memory in Language Comprehension , 1988 .

[14]  S. Schwartz.  Measuring Reading Competence , 1984 .

[15]  Herbert A. Simon,et al.  Models of Competence in Solving Physics Problems , 1980, Cogn. Sci..

[16]  A. Miyake,et al.  Models of Working Memory: Mechanisms of Active Maintenance and Executive Control , 1999 .

[17]  E. Gibson.  Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[18]  Slava Kalyuga,et al.  Learner Experience and Efficiency of Instructional Guidance , 2001 .

[19]  J. Sweller.  Evolution of human cognitive architecture , 2003 .

[20]  M. Wittrock.  Focus on educational psychology , 1967 .

[21]  Walter Kintsch,et al.  Comprehension: A Paradigm for Cognition , 1998 .

[22]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[23]  G. Waters,et al.  Age, working memory, and on-line syntactic processing in sentence comprehension. , 2001, Psychology and aging.

[24]  Randy Elliot Bennett,et al.  Item generation and beyond: Applications of schema theory to mathematics assessment. , 2002 .

[25]  J. B. Wyman,et al.  What is reading ability , 1921 .

[26]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[27]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[28]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[29]  Stephani Foraker,et al.  Memory structures that subserve sentence comprehension , 2003 .

[30]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .