Calibration Research: Where Do We Go from Here?

Research on calibration remains a popular line of inquiry. Calibration is the degree of fit between a person’s judgment of performance and his or her actual performance. Given the continued interest in this topic, the questions posed in this article are fruitful directions to pursue to help address gaps in calibration research. In this article, we have identified six research directions that if productively pursued, could greatly expand our knowledge of calibration. The six research directions are: (a) what are the effects of varying the anchoring mechanisms from which calibration judgments are made, (b) how does calibration accuracy differ as a function of incentives and task authenticity, (c) how do students self-report the basis of their calibration judgments, (d) how do group interactions and social comparisons affect calibration accuracy, (e) what is the relation between absolute and relative accuracy, and (f) to what extent does calibration accuracy predict achievement? To help point the way to where we go from here in calibration research, we provide these research questions, propose research methods designed to address them, and identify prior, related studies that have shown promise in leading the way to fill these gaps in the literature.

[1]  Philip H. Winne,et al.  Improving Measurements of Self-Regulated Learning , 2010 .

[2]  Linda Bol,et al.  The effects of individual or group guidelines on the calibration accuracy and achievement of high school biology students , 2012 .

[3]  B. Zimmerman.  Investigating Self-Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects , 2008, American Educational Research Journal.

[4]  Douglas J. Hacker,et al.  A Comparison of the Effects of Practice Tests and Traditional Review on Performance and Calibration , 2001 .

[5]  Maria T. Potenza,et al.  Constraints on the Calibration of Performance , 1993 .

[6]  John Dunlosky,et al.  Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention , 2012 .

[7]  A. Koriat,et al.  Comparing objective and subjective learning curves: judgments of learning exhibit increased underconfidence with practice. , 2002, Journal of experimental psychology. General.

[8]  Christopher Hertzog,et al.  Metacognitive influences on study time allocation in an associative recognition task: An analysis of adult age differences. , 2009, Psychology and aging.

[9]  Li Cao,et al.  Metacognitive Monitoring Accuracy and Student Performance in the Postsecondary Classroom. , 2005 .

[10]  G. Keren.  Calibration and probability judgements: Conceptual and methodological issues , 1991 .

[11]  Douglas J. Hacker,et al.  The Influence of Overt Practice, Achievement Level, and Explanatory Style on Calibration Accuracy and Performance , 2005 .

[12]  Bracha Kramarski,et al.  Enhancing Mathematical Reasoning in the Classroom: The Effects of Cooperative Learning and Metacognitive Training , 2003 .

[13]  B. Zimmerman,et al.  Self-regulation: Where metacognition and motivation intersect. , 2009 .

[14]  Linda Bol,et al.  Metacognition in education: A focus on calibration. , 2008 .

[15]  ERNEST CRAWLEY,et al.  Social Origins , 1941, Nature.

[16]  Li Cao,et al.  The effect of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self-efficacy , 2006 .

[17]  Masamichi Yuzawa,et al.  The Effects of Social Cues on Confidence Judgments Mediated by Knowledge and Regulation of Cognition , 2001 .

[18]  Michael Pressley,et al.  Self-Regulated Learning: Monitoring Learning From Text , 1990 .

[19]  Tracy Linderholm,et al.  Anchoring effects on prospective and retrospective metacomprehension judgments as a function of peer performance information , 2011 .

[20]  B. Zimmerman.  Attaining self-regulation: A social cognitive perspective. , 2000 .

[21]  Linda Bol,et al.  Explaining calibration accuracy in classroom contexts: the effects of incentives, reflection, and explanatory style , 2008 .

[22]  Ruth H. Maki,et al.  Individual Differences in Absolute and Relative Metacomprehension Accuracy. , 2005 .

[23]  T. O. Nelson,et al.  Anchoring Effects in the Absolute Accuracy of Immediate versus Delayed Judgments of Learning. , 2004 .

[24]  Daniel L. Dinsmore,et al.  What are confidence judgments made of? Students' explanations for their confidence ratings and what that means for calibration , 2013 .

[25]  Henrik Olsson,et al.  Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. , 2000 .

[26]  T. O. Nelson,et al.  Lack of pervasiveness of the underconfidence-with-practice effect: boundary conditions and an explanation via anchoring. , 2005, Journal of experimental psychology. General.

[27]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[28]  N. Epley,et al.  When effortful thinking influences judgmental anchoring: differential effects of forewarning and incentives on self‐generated and externally provided anchors , 2005 .

[29]  J. Metcalfe,et al.  Evidence that judgments of learning are causally related to study choice , 2008, Psychonomic bulletin & review.

[30]  Thomas Gilovich,et al.  Are Adjustments Insufficient? , 2004, Personality & social psychology bulletin.

[31]  B. Zimmerman,et al.  Social Origins of Self-Regulatory Competence , 1997 .

[32]  Ruth H. Maki,et al.  The basis of test predictions for text material. , 1992 .

[33]  Thomas Mussweiler,et al.  Overcoming the Inevitable Anchoring Effect: Considering the Opposite Compensates for Selective Accessibility , 2000 .

[34]  Paul W. Grimes,et al.  The Overconfident Principles of Economics Student: An Examination of a Metacognitive Skill , 2002 .

[35]  Daniel C. Moos,et al.  Measuring Cognitive and Metacognitive Regulatory Processes During Hypermedia Learning: Issues and Challenges , 2010 .

[36]  P. Juslin,et al.  Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. , 2000, Psychological review.

[37]  T. O. Nelson,et al.  Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not of the absolute performance on an individual item. Comments on Schraw (1995) , 1996 .

[38]  B. Kramarski,et al.  Group-Metacognitive Support for Online Inquiry in Mathematics with Differential Self-Questioning , 2009 .

[39]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[40]  Linda Bol,et al.  Test prediction and performance in a classroom context. , 2000 .