The effect of metacognitive monitoring feedback on performance in a computer-based training simulation.

This laboratory experiment was designed to study the effect of metacognitive monitoring feedback on performance in a computer-based training simulation. According to prior research on metacognition, the accurate checking of learning is a critical part of improving the quality of human performance. However, only rarely have researchers studied the learning effects of the accurate checking of retrospective confidence judgments (RCJs) during a computer-based military training simulation. In this study, we provided participants feedback screens after they had completed a warning task and identification task in a radar monitoring simulation. There were two groups in this experiment. One group (group A) viewed the feedback screens with the flight path of all target aircraft and the triangular graphs of both RCJ scores and human performance together. The other group (group B) only watched the feedback screens with the flight path of all target aircraft. There was no significant difference in performance improvement between groups A and B for the warning task (Day 1: group A - 0.347, group B - 0.305; Day 2: group A - 0.488, group B - 0.413). However, the identification task yielded a significant difference in performance improvement between these groups (Day 1: group A - 0.174, group B - 0.1555; Day 2: group A - 0.324, group B - 0.199). The results show that debiasing self-judgment of the identification task produces a positive training effect on learners. The findings of this study will be beneficial for designing an advanced instructional strategy in a simulation-based training environment.

[1]  Linda Bol,et al.  Metacognition in education: A focus on calibration. , 2008 .

[2]  Arathi Sethumadhavan,et al.  Knowing What You Know , 2011 .

[3]  Gretchen A. Macht,et al.  Comparison of Individual and Team-Based Dynamic Decision-Making Task (Anti-Air Warfare Coordinator): Consideration of Subjective Mental Workload Metacognition , 2012 .

[4]  Philip H. Winne,et al.  Students’ calibration of knowledge and learning processes: Implications for designing powerful software learning environments , 2004 .

[5]  Li Cao,et al.  Metacognitive Monitoring Accuracy and Student Performance in the Postsecondary Classroom. , 2005 .

[6]  Bradford S. Bell,et al.  Advances in Technology-Based Training , 2007 .

[7]  T. O. Nelson,et al.  Why investigate metacognition , 1994 .

[8]  Kerri L. Johnson,et al.  Why People Fail to Recognize Their Own Incompetence , 2003 .

[9]  Ling Rothrock,et al.  Applying Fuzzy Linear Regression to Understand Metacognitive Judgments in a Human-in-the-Loop Simulation Environment , 2016, IEEE Transactions on Human-Machine Systems.

[10]  Logan Fiorella,et al.  Training for Collaborative Problem Solving , 2014 .

[11]  Jessica L. Gosney,et al.  Metacognitive Judgments in a Simulated Luggage Screening Task , 2005 .

[12]  Colleen M. Kelley,et al.  Solve the problem first: constructive solution strategies can influence the accuracy of retrospective confidence judgments. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[13]  Jennifer J. Vogel-Walcutt,et al.  Differential impact of two types of metacognitive prompting provided during simulation-based training , 2012, Comput. Hum. Behav..

[14]  N. Purdie,et al.  Effects of Learning Skills Interventions on Student Learning: A Meta-Analysis , 1996 .

[15]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[16]  T. O. Nelson,et al.  When People's Judgments of Learning (JOLs) are Extremely Accurate at Predicting Subsequent Recall: The “Delayed-JOL Effect” , 1991 .

[17]  Gregory Schraw.  Promoting general metacognitive awareness , 1998 .

[18]  Jennifer J. Vogel-Walcutt,et al.  Instructional strategies framework for military training systems , 2013, Comput. Hum. Behav..

[19]  Fred Paas,et al.  Principles for Designing Effective and Efficient Training of Complex Cognitive Skills , 2009 .

[20]  Louis Narens,et al.  Using the past to predict the future , 2005, Memory & cognition.

[21]  E. Salas,et al.  Learning and Performance Across Disciplines: An Epilogue for Moving Multidisciplinary Research Toward an Interdisciplinary Science of Expertise , 2008 .

[22]  John Dunlosky,et al.  Handbook of Metamemory and Memory , 2013 .

[23]  J. Flavell.  Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .

[24]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[25]  Robert M. Klassen,et al.  A Question of Calibration: A Review of the Self-Efficacy Beliefs of Students with Learning Disabilities , 2002 .

[26]  Wen-Hwa Ko,et al.  A study of the relationships among effective learning, professional competence, and learning performance in culinary field , 2012 .

[27]  H. Walberg,et al.  What Influences Learning? A Content Analysis of Review Literature , 1990 .

[28]  Linda Bol,et al.  Test prediction and performance in a classroom context. , 2000 .

[29]  T. O. Nelson.  Metamemory: A Theoretical Framework and New Findings , 1990 .

[30]  Philip H. Winne,et al.  Exploring students’ calibration of self reports about study tactics and achievement , 2002 .

[31]  James P. Van Overschelde.  Metacognition: Knowing about knowing. , 2008 .

[32]  Robert M. Gagné,et al.  The Conditions of Learning and Theory of Instruction , 1985 .

[33]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[34]  Linda Bol,et al.  The effects of individual or group guidelines on the calibration accuracy and achievement of high school biology students , 2012 .

[35]  P. Pintrich,et al.  Handbook of self-regulation , 2000 .

[36]  Douglas J. Hacker,et al.  A Comparison of the Effects of Practice Tests and Traditional Review on Performance and Calibration , 2001 .

[37]  Jennifer J. Vogel-Walcutt,et al.  Metacognitive Prompting as a Generalizable Instructional Tool in Simulation-Based Training , 2011 .

[38]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[39]  Gregory Schraw,et al.  Assessing metacognitive awareness , 1994 .

[40]  Stephen M. Fiore,et al.  Fostering constructive cognitive and metacognitive activity in computer-based complex task training environments , 2004, Comput. Hum. Behav..

[41]  Li Cao,et al.  The effect of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self-efficacy , 2006 .

[42]  Douglas J. Hacker,et al.  Handbook of Metacognition in Education , 2009 .

[43]  Gideon Keren,et al.  Cognitive Aids and Debiasing Methods: CAN Cognitive Pills Cure Cognitive Ills? , 1990 .

[44]  John L. Nietfeld,et al.  Using strategy instruction and confidence judgments to improve metacognitive monitoring , 2009 .

[45]  Petros Georghiades,et al.  From the general to the situated: three decades of metacognition , 2004 .

[46]  Jennifer J. Vogel-Walcutt,et al.  Making Metacognition Explicit: Developing a Theoretical Foundation for Metacognitive Prompting During Scenario-based Training , 2010 .

[47]  S. A. Coutinho,et al.  Self-efficacy, metacognition, and performance. , 2008 .

[48]  T. O. Nelson,et al.  Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect , 1992, Memory & cognition.

[49]  L. Rothrock *,et al.  A theoretical framework and quantitative architecture to assess team task complexity in dynamic environments , 2005 .

[50]  Ming Ming Chiu,et al.  Relations of mathematics self-concept and its calibration with mathematics achievement: Cultural differences among fifteen-year-olds in 34 countries , 2010 .

[51]  Ling Rothrock,et al.  Using Time Windows to Evaluate Operator Performance , 2001 .

[52]  Jung Hyup Kim,et al.  Investigating the Effects of Metacognition in Dynamic Control Tasks , 2011, HCI.

[53]  R. Azevedo.  Using Hypermedia as a Metacognitive Tool for Enhancing Student Learning? The Role of Self-Regulated Learning , 2005 .