Measuring Self-Regulation in Computer-Based Learning Environments

I provide a summary of the four invited articles in this special issue and compare and contrast different methods for measuring self-regulation in computer-based learning environments (CBLEs). I present a taxonomy that distinguishes between offline and online measures and further distinguishes subcategories within each of these categories. I discuss four measurement challenges, including orchestrating a repertoire of automated assessments to capture these ongoing activities, developing scoring rubrics and scoring criteria, interpreting assessments, and aligning multiple outcome measures. I also discuss how measures of self-regulated learning in CBLEs can be used to assess important cognitive processes such as planning, strategy use, monitoring, and increased learning. I conclude with several suggestions for measurement practices that enhance and further strengthen the validity of research conclusions.

[1]  Gregory Schraw,et al.  Assessing Teachers' Epistemological and Ontological Worldviews , 2008 .

[2]  Bruce R Reichenbach,et al.  Introduction to Critical Thinking , 2000 .

[3]  Scotty D. Craig,et al.  Promoting Constructive Activities that Support Vicarious Learning During Computer-Based Instruction , 2006 .

[4]  P. Alexander.  The Development of Expertise: The Journey From Acclimation to Proficiency , 2003 .

[5]  John C. Nesbit,et al.  Supporting Self-Regulated Learning with Cognitive Tools , 2009 .

[6]  Gregory Schraw,et al.  Assessing metacognitive awareness , 1994 .

[7]  M. Ashcraft,et al.  Telling stories: the perils and promise of using verbal reports to study math strategies. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[9]  Morten Hertzum,et al.  Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload? , 2009, Behav. Inf. Technol..

[10]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[11]  Madhucchanda Sen.  An Introduction to Critical Thinking , 2010 .

[12]  Stuart A. Karabenick,et al.  Assessing Academic Self-Regulated Learning , 2005 .

[13]  Gregory Schraw,et al.  Consistency and development of teachers’ epistemological and ontological world views , 2010 .

[14]  Arthur C. Graesser,et al.  Self-Regulated Learning in Learning Environments With Pedagogical Agents That Interact in Natural Language , 2010 .

[15]  Roger Azevedo,et al.  A macro-level analysis of SRL processes and their relations to the acquisition of a sophisticated mental model of a complex system☆☆☆ , 2009 .

[16]  Philip H. Winne,et al.  Self-regulated learning viewed from models of information processing. , 2001 .

[17]  Danielle S. McNamara,et al.  iSTART: A Web-based tutor that teaches self-explanation and metacognitive reading strategies. , 2007 .

[18]  Stephanie Pieschl,et al.  The Role of Epistemic Beliefs in Students’ Self-Regulated Learning With Computer-Based Learning Environments: Conceptual and Methodological Issues , 2010 .

[19]  D. Nicol.  Assessment for learner self‐regulation: enhancing achievement in the first year using learning technologies , 2009 .

[20]  M. Pressley,et al.  Cognitive Strategies Instruction: From Basic Research to Classroom Instruction , 2009 .

[21]  R. Azevedo,et al.  A Theoretical Review of Winne and Hadwin’s Model of Self-Regulated Learning: New Perspectives and Directions , 2007 .

[22]  Philip H. Winne,et al.  The weave of motivation and self-regulated learning. , 2008 .

[23]  Douglas F. Kauffman,et al.  Self-regulated learning theory , 2002 .

[24]  V. Aleven,et al.  Help Seeking and Help Design in Interactive Learning Environments , 2003 .

[25]  K. VanLehn,et al.  Scaffolding Deep Comprehension Strategies Through Point&Query, AutoTutor, and iSTART , 2005 .

[26]  B. Zimmerman,et al.  Competence and Control Beliefs: Distinguishing the Means and Ends , 2006 .

[27]  B. Zimmerman,et al.  Self-regulated learning and academic achievement : theoretical perspectives , 2001 .

[28]  Philip H. Winne,et al.  Measuring Self-Regulated Learning , 2000 .

[29]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[30]  Arthur C. Graesser,et al.  Agent Technologies Designed to Facilitate Interactive Knowledge Construction , 2008 .

[31]  Douglas J. Hacker,et al.  Handbook of Metacognition in Education , 2009 .

[32]  Marlene Schommer.  Effects of beliefs about the nature of knowledge on comprehension. , 1990 .

[33]  Danielle S. McNamara.  Reading comprehension strategies : theories, interventions, and technologies , 2007 .

[34]  R. Azevedo.  Using Hypermedia as a Metacognitive Tool for Enhancing Student Learning? The Role of Self-Regulated Learning , 2005 .

[35]  Howard Eisner,et al.  Information Age , 2008, Wiley Encyclopedia of Computer Science and Engineering.

[36]  Gregory Schraw.  Knowledge: Structures and Processes. , 2006 .

[37]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[38]  K. Koedinger,et al.  Automated, Unobtrusive, Action-by-Action Assessment of Self-Regulation During Learning With an Intelligent Tutoring System , 2010 .

[39]  D. Kuhn.  THE SKILLS OF ARGUMENT , 2008, Education for Thinking.

[40]  Stephen K. Reed,et al.  Cognitive Architectures for Multimedia Learning , 2006 .

[41]  Daniel C. Moos,et al.  Measuring Cognitive and Metacognitive Regulatory Processes During Hypermedia Learning: Issues and Challenges , 2010 .

[42]  John C. Nesbit,et al.  Methodological Issues in Educational Psychology , 2006 .

[43]  Roger Azevedo,et al.  Understanding the complex nature of self-regulatory processes in learning with computer-based learning environments: an introduction , 2007 .

[44]  B. Zimmerman.  Theories of Self-Regulated Learning and Academic Achievement: An Overview and Analysis , 2001 .

[45]  K. A. Ericsson,et al.  Protocol Analysis and Expert Thought: Concurrent Verbalizations of Thinking during Experts' Performance on Representative Tasks , 2006 .

[46]  J. Frederiksen,et al.  A Theoretical Framework and Approach for Fostering Metacognitive Development , 2005 .

[47]  P. Pintrich.  The role of goal orientation in self-regulated learning. , 2000 .

[48]  B. Zimmerman.  Investigating Self-Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects , 2008, American Educational Research Journal.

[49]  B. Zimmerman.  Attaining self-regulation: A social cognitive perspective. , 2000 .

[50]  Roger Azevedo,et al.  Teaching and Learning in Technology-Rich Environments , 2006 .

[51]  Gregory Schraw,et al.  The use of computer-based environments for understanding and improving self-regulation , 2007 .

[52]  K. Moore,et al.  What do children need to flourish? : conceptualizing and measuring indicators of positive development , 2005 .

[53]  Paul R. Pintrich,et al.  Development and Validation of the Epistemic Belief Inventory (EBI) , 2001 .

[54]  Gregory Schraw,et al.  Teachers’ Epistemological World Views and Educational Practices , 2003, Journal of Cognitive Education and Psychology.