Are the unskilled doomed to remain unaware

The unskilled-and-unaware problem describes a negative relationship between one’s skill level and self-assessment bias: the less skilled are, on average, more unaware of the absolute and relative quality of their performance. In this paper, we study whether, and to what extent, the miscalibration (largely, overconfidence) of the unskilled can be reduced by feedback. We report the results of two studies, one in a natural setting and one in a more controlled setting, where participants make incentivized judgments of their absolute and relative performance in various tasks and feedback conditions. In the first study, participants improve their calibration after being exposed to naturally available information in the form of environmental feedback (i.e., feedback about the nature of the task) and calibration feedback (i.e., feedback about one’s absolute and relative performance), but it is impossible to separate the effects of the two types of feedback. In the more controlled setting of the second study, we identified a positive effect of calibration feedback alone. In both studies, it is the unskilled who improve their calibration most. Our results suggest that the unskilled may not be doomed to be especially unaware. We also identify an important difference between the effects of feedback on the calibration of absolute and relative performance judgments. While the calibration of absolute performance judgments is more uniformly amenable to feedback, there appears to be a residual miscalibration of relative performance judgments by the unskilled that we attribute to differences in information acquisition and self-image.

[1]  R. Hertwig,et al.  Experimental practices in economics: A methodological challenge for psychologists? , 2001, Behavioral and Brain Sciences.

[2]  Andreas Ortmann,et al.  How financial incentives and cognitive abilities affect task performance in laboratory settings: an illustration † , 2004 .

[3]  Stone,et al.  Training to Improve Calibration and Discrimination: The Effects of Performance and Environmental Feedback. , 2000, Organizational behavior and human decision processes.

[4]  J. Suls,et al.  Flawed Self-Assessment , 2004, Psychological science in the public interest : a journal of the American Psychological Society.

[5]  Craig R. M. McKenzie,et al.  Underweighting Alternatives and Overconfidence , 1997 .

[6]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[7]  Martin Strobel,et al.  Real effort versus chosen effort in experiments , 2007 .

[8]  John R. Nofsinger,et al.  Trading Performance, Disposition Effect, Overconfidence, Representativeness Bias, and Experience of Emerging Market Investors , 2007 .

[9]  Colin Camerer,et al.  The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework , 1999 .

[10]  Kerri L. Johnson,et al.  Why the Unskilled are Unaware: Further Explorations of (Absent) Self-Insight Among the Incompetent , 2006, Organizational behavior and human decision processes.

[11]  Jeffrey M. Wooldridge,et al.  Solutions Manual and Supplementary Materials for Econometric Analysis of Cross Section and Panel Data , 2003 .

[12]  Argie Leach,et al.  Assessing the Competence of Specimen-Processing Personnel , 2000 .

[13]  J. Yates,et al.  General knowledge overconfidence : Cross-national variations, response style, and reality , 1997 .

[14]  Andreas Ortmann,et al.  Are the Unskilled Really that Unaware? An Alternative Explanation , 2007 .

[15]  Winston R. Sieck,et al.  The recalcitrance of overconfidence and its contribution to decision aid neglect , 2005 .

[16]  Gideon Keren,et al.  On The Calibration of Probability Judgments: Some Critical Comments and Alternative Perspectives , 1997 .

[17]  Olesya Govorun,et al.  Better-than-Average Effect , 2020, Encyclopedia of Personality and Individual Differences.

[18]  J. Klayman,et al.  Judgments of Performance: The Relative, the Absolute, and the In-Between , 2005 .

[19]  Johnnie E. V. Johnson,et al.  Calibration of Subjective Probability Judgments in a Naturalistic Setting. , 2001, Organizational behavior and human decision processes.

[20]  B. Kőszegi.  EGO UTILITY, OVERCONFIDENCE, AND TASK CHOICE , 2006 .

[21]  Aldo Rustichini,et al.  Overconfident: Do You Put Your Money on it? , 2005 .

[22]  A. H. Murphy,et al.  Probability Forecasting in Meteorology , 1984 .

[23]  J. Kruger,et al.  Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. , 1999, Journal of personality and social psychology.

[24]  Lawrence D. Phillips,et al.  Cultural Differences in Probabilistic Thinking , 1978 .

[25]  Briony D. Pulford,et al.  Overconfidence : Feedback and item difficulty effects , 1997 .

[26]  Shawn P. Curley,et al.  Probability judgment accuracy for general knowledge. Cross‐national differences and assessment methods , 1995 .

[27]  Don A. Moore,et al.  Overconfidence and Underconfidence: When and Why People Underestimate (and Overestimate) the Competition , 2007 .

[28]  Linda Bol,et al.  Test prediction and performance in a classroom context. , 2000 .

[29]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[30]  David Dunning,et al.  Self-Insight: Roadblocks and Detours on the Path to Knowing Thyself , 2005 .

[31]  D. Moore.  When good = better than average , 2007, Judgment and Decision Making.

[32]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[33]  Asher Koriat,et al.  Monitoring and control processes in the strategic regulation of memory accuracy , 1996 .

[34]  Colin Camerer,et al.  Overconfidence and Excess Entry: An Experimental Approach , 1999 .

[35]  Ulrike Malmendier,et al.  CEO Overconfidence and Corporate Investment , 2002 .

[36]  Dirk Engelmann,et al.  The False Consensus Effect Disappears if Representative Information and Monetary Incentives Are Given , 2000 .

[37]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[38]  Janet A. Sniezek,et al.  Confidence depends on level of aggregation , 1991 .

[39]  P. Juslin,et al.  Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. , 2000, Psychological review.

[40]  D. Moore,et al.  The trouble with overconfidence. , 2008, Psychological review.

[41]  Magnus Johannesson,et al.  Confidence interval estimation tasks and the economics of overconfidence , 2003 .

[42]  Mandeep K. Dhami,et al.  The role of representative design in an ecological approach to cognition. , 2004, Psychological bulletin.

[43]  F. Dumont,et al.  Eliminating Overconfidence in Psychodiagnosis: Strategies for Training and Practice , 1997 .

[44]  Uwe Sunde,et al.  Self-Confidence and Search , 2006, SSRN Electronic Journal.

[45]  Steven D. Penrod,et al.  Performance feedback improves the resolution of confidence judgments , 1988 .

[46]  B. Fischhoff,et al.  Training for calibration. , 1980 .

[47]  Paul J. Ferraro,et al.  Know Thyself: Competence and Self-awareness , 2010 .

[48]  F. Bolger,et al.  Market entry decisions: effects of absolute and relative confidence. , 2008, Experimental psychology.

[49]  Winston R. Sieck,et al.  Option fixation: A cognitive contributor to overconfidence , 2007 .

[50]  Joachim Krueger,et al.  Unskilled, unaware, or both? The better-than-average heuristic and statistical regression predict errors in estimates of own performance. , 2002, Journal of personality and social psychology.

[51]  Richard P. Larrick,et al.  Skilled or unskilled, but still unaware of it: how perceptions of difficulty drive miscalibration in relative comparisons. , 2006, Journal of personality and social psychology.