Overconfidence in personnel selection: When and why unstructured interview information can hurt hiring decisions

Overconfidence is an important bias related to the ability to recognize the limits of one’s knowledge. The present study examines overconfidence in predictions of job performance for participants presented with information about candidates based solely on standardized tests versus those who also were presented with unstructured interview information. We conducted two studies with individuals responsible for hiring decisions. Results showed that individuals presented with interview information exhibited more overconfidence than individuals presented with test scores only. In a third study, consisting of a betting competition for undergraduate students, larger overconfidence was related to fewer payoffs. These combined results emphasize the importance of studying confidence and decision-related variables in selection decisions. Furthermore, while previous research has shown that the predictive validity of unstructured interviews is low, this study provides compelling evidence that they not only fail to help personnel selection decisions, but can actually hurt them.

[1]  J. Kagan,et al.  Rational choice in an uncertain world , 1988 .

[2]  J. Frank Yates.  Culture and Probability Judgment , 2010 .

[3]  Sniezek,et al.  The Social Influence of Confidence in Group Decision Making , 1997, Journal of experimental social psychology.

[4]  J. Farr,et al.  Handbook of employee selection , 2010 .

[5]  D. Moore,et al.  The trouble with overconfidence. , 2008, Psychological review.

[6]  Kenneth G. Brown,et al.  Seven common misconceptions about human resource practices: Research findings versus practitioner beliefs , 2002 .

[7]  N. Schmitt.  The Oxford Handbook of Personnel Assessment and Selection , 2012 .

[8]  N. Kuncel,et al.  Complex Predictions and Assessor Mystique , 2011, Industrial and Organizational Psychology.

[9]  Ray W. Cooksey,et al.  Judgment analysis : theory, methods, and applications , 1996 .

[10]  Margaret E. Brooks,et al.  Are common language effect sizes easier to understand than traditional effect sizes? , 2014, The Journal of applied psychology.

[11]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[12]  Stephanie C. Payne,et al.  The incremental validity of interview scores over and above cognitive ability and conscientiousness scores , 2000 .

[13]  Reid Hastie,et al.  Calibration Trumps Confidence as a Basis for Witness Credibility , 2006, Psychological science.

[14]  Don A. Moore,et al.  Cheap talk and credibility: The consequences of confidence and accuracy on advisor credibility and persuasiveness , 2013 .

[15]  P. Juslin,et al.  Judgment and decision making: Neo-Brunswikian and process tracing approaches , 1999 .

[16]  Gerd Gigerenzer,et al.  Communicating Statistical Information , 2000, Science.

[17]  Dilek Önkal,et al.  Professional vs Amateur Judgment Accuracy: The Case of Foreign Exchange Rates , 2003 .

[18]  Stephen J. Hoch,et al.  Outcome feedback: hindsight and information , 1989 .

[19]  P. Juslin,et al.  Format dependence in subjective probability calibration , 1999 .

[20]  A. Hyde.  Interviews , 2016 .

[21]  Philip E. Tetlock,et al.  Superforecasting: The Art and Science of Prediction , 2015 .

[22]  A. Bakker,et al.  Why are structured interviews so rarely used in personnel selection? , 2002, The Journal of applied psychology.

[23]  Briony D. Pulford,et al.  Overconfidence : Feedback and item difficulty effects , 1997 .

[24]  Nigel Harvey,et al.  Blackwell Handbook of Judgment and Decision Making , 2004 .

[25]  Roger Ratcliff,et al.  Modeling confidence judgments, response times, and multiple choices in decision making: recognition memory and motion discrimination. , 2013, Psychological review.

[26]  Winston R. Sieck,et al.  Cross-Cultural Variations in Probability Judgment Accuracy: Beyond General Knowledge Overconfidence? , 1998, Organizational behavior and human decision processes.

[27]  James G. Field,et al.  Correlational effect size benchmarks. , 2015, The Journal of applied psychology.

[28]  Scott Highhouse.  Stubborn Reliance on Intuition and Subjectivity in Employee Selection , 2008, Industrial and Organizational Psychology.

[29]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[30]  Steven D. Penrod,et al.  Performance feedback improves the resolution of confidence judgments , 1988 .

[31]  Mike W. Peng,et al.  Joint venture dissolution as corporate divorce , 2002 .

[32]  B. Fischhoff,et al.  Subjective Confidence in Forecasts. , 1982 .

[33]  Adam Wierman,et al.  Thinking Fast and Slow , 2017, SIGMETRICS Perform. Evaluation Rev..

[34]  Ronald J. Ebert,et al.  Policy capturing in the employment interview. , 1986 .

[35]  S. Oskamp.  OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[36]  R. Hogarth,et al.  Determinants of Linear Judgment: A Meta-Analysis of Lens Model Studies , 2007, Psychological bulletin.

[37]  Derek J. Koehler,et al.  Modeling patterns of probability calibration with random support theory: Diagnosing case-based judgment ☆ , 2005 .

[38]  Josephine Sosa-Fey.  Essentials of Personnel Assessment and Selection , 2007 .

[39]  Timothy J. Pleskac,et al.  Two-stage dynamic signal detection: a theory of choice, decision time, and confidence. , 2010, Psychological review.

[40]  A. Hayes.  PROCESS : A Versatile Computational Tool for Observed Variable Mediation , Moderation , and Conditional Process Modeling 1 , 2012 .

[41]  J A Swets,et al.  Better decisions through science. , 2000, Scientific American.

[42]  C. M. Allwood,et al.  Old and very old adults as witnesses: event memory and metamemory , 2015 .

[43]  David L. Ronis,et al.  Components of probability judgment accuracy: Individual consistency and effects of subject matter and assessment method. , 1987 .

[44]  Winston R. Sieck,et al.  The recalcitrance of overconfidence and its contribution to decision aid neglect , 2005 .

[45]  Scott Highhouse.  ASSESSING THE CANDIDATE AS A WHOLE: A HISTORICAL AND CRITICAL ANALYSIS OF INDIVIDUAL PSYCHOLOGICAL ASSESSMENT FOR PERSONNEL DECISION MAKING , 2002 .

[46]  R. Scheines,et al.  Organizational Behavior and Human Decision Processes , 1977 .

[47]  Albert E. Mannes,et al.  A Behavioral Demonstration of Overconfidence in Judgment , 2013, Psychological science.

[48]  Eduardo Barros,et al.  Using General Mental Ability and Personality Traits to Predict Job Performance in Three Chilean Organizations , 2014 .

[49]  Robin M. Hogarth,et al.  The challenge of representative design in psychology and economics , 2004 .

[50]  Steven D. Maurer,et al.  The validity of employment interviews: A comprehensive review and meta-analysis. , 1994 .

[51]  R. Dawes,et al.  Belief in the unstructured interview: The persistence of an illusion , 2013, Judgment and Decision Making.

[52]  Ulrike Malmendier,et al.  Behavioral CEOs: The Role of Managerial Overconfidence , 2015 .

[53]  W. Revelle.  psych: Procedures for Personality and Psychological Research , 2017 .

[54]  Scott Highhouse.  Designing Experiments That Generalize , 2009 .

[55]  F. Schmidt,et al.  The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. , 1998 .

[56]  D. Bowen,et al.  Understanding HRM–Firm Performance Linkages: The Role of the “Strength” of the HRM System , 2004 .

[57]  Murray R. Barrick,et al.  Personality and Performance at the Beginning of the New Millennium: What Do We Know and Where Do We Go Next? , 2001 .

[58]  Nathan R. Kuncel,et al.  Some New (and Old) Suggestions for Improving Personnel Selection , 2008, Industrial and Organizational Psychology.

[59]  Melissa Woodard Barringer,et al.  A Review and Analysis of the Policy-Capturing Methodology in Organizational Research: Guidelines for Research and Practice , 2002 .

[60]  Esther Kaufmann,et al.  A Critical Meta-Analysis of Lens Model Studies in Human Judgment and Decision-Making , 2013, PloS one.

[61]  C. Bartlett,et al.  Clinical prediction: Does one sometimes know too much? , 1966 .

[62]  Thomas R. Stewart,et al.  Improving Reliability of Judgmental Forecasts , 2001 .

[63]  R I Benjamin,et al.  Critical IT (information technology) issues: the next ten years. , 1992, Sloan management review.

[64]  Edgar E. Kausel,et al.  Employee Selection Decisions , 2013 .

[65]  Janet L Barnes-Farrell,et al.  Development and initial evaluation of an enhanced measure of boundary flexibility for the work and family domains. , 2010, Journal of occupational health psychology.

[66]  C. Stevens,et al.  ANTECEDENTS OF INTERVIEW INTERACTIONS, INTERVIEWERS' RATINGS, AND APPLICANTS' REACTIONS , 1998 .

[67]  B. Fischhoff,et al.  Judgment and decision making. , 2012, Wiley interdisciplinary reviews. Cognitive science.

[68]  Reid Hastie,et al.  Effects of Amount of Information on Judgment Accuracy and Confidence , 2008 .

[69]  Richard J. Klimoski,et al.  Intuiting the Selection Context , 2008, Industrial and Organizational Psychology.

[70]  James M. LeBreton,et al.  Relative Importance Analysis: A Useful Supplement to Regression Analysis , 2011 .

[71]  Nigel Harvey,et al.  Confidence in judgment , 1997, Trends in Cognitive Sciences.

[72]  G. Dagnino,et al.  The Origin of Failure: A Multidisciplinary Appraisal of the Hubris Hypothesis and Proposed Research Agenda , 2014 .

[73]  A. Todorov,et al.  The illusion of knowledge: When more information reduces accuracy and increases confidence , 2007 .

[74]  J. W. Johnson.  A Heuristic Method for Estimating the Relative Weight of Predictor Variables in Multiple Regression , 2000, Multivariate behavioral research.

[75]  Jiddu Krishnamurti,et al.  On Self-Knowledge , 2002 .

[76]  Sun Young Lee,et al.  Discrimination in selection decisions: Integrating stereotype fit and interdependence theories , 2014 .

[77]  L. Hough,et al.  Personality : Its Measurement and Validity for Employee Selection , 2010 .

[78]  Frederick L. Oswald,et al.  Personality Testing and Industrial–Organizational Psychology: Reflections, Progress, and Prospects , 2008, Industrial and Organizational Psychology.

[79]  T. Sarbin.  A Contribution to the Study of Actuarial and Individual Methods of Prediction , 1943, American Journal of Sociology.

[80]  Neal Schmitt,et al.  Personality and Cognitive Ability as Predictors of Effective Performance at Work , 2014 .

[81]  John P. Hausknecht,et al.  Applicant Reactions to Selection Procedures: An Updated Model and Meta-Analysis , 2004 .

[82]  Darryl A. Seale,et al.  Sequential Decision Making with Relative Ranks: An Experimental Investigation of the "Secretary Problem"> , 1997 .

[83]  Lynda Aiman-Smith,et al.  Conducting Studies of Decision Making in Organizational Contexts: A Tutorial for Policy-Capturing and Other Regression-Based Techniques , 2002 .

[84]  Winfred Arthur,et al.  Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs. , 1994 .

[85]  Filip Lievens,et al.  The importance of traits and abilities in supervisors' hirability decisions as a function of method of assessment , 2005 .

[86]  C. González-Vallejo,et al.  Assessment of calibration for reconstructed eye-witness memories. , 2009, Acta psychologica.

[87]  Kimberly A. Eddleston,et al.  Moderators of the Relationship Between Work-Family Conflict and Career Satisfaction , 2002 .

[88]  D. Terpstra,et al.  Why Some Potentially Effective Staffing Practices are Seldom Used , 1997 .

[89]  A. Goodie.  The effects of control on betting: paradoxical betting on items of high confidence with low value. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[90]  L. Farrand.  THE AMERICAN PSYCHOLOGICAL ASSOCIATION. , 1897, Science.

[91]  A. Tenbrunsel,et al.  Job-decision inconsistencies involving social comparison information: the role of dominating alternatives. , 2002, The Journal of applied psychology.

[92]  James Ramirez,et al.  Good probabilistic forecasters: The ‘consumer's’ perspective , 1996 .

[93]  Lyle Brenner,et al.  Perspectives on Probability Judgment Calibration , 2008 .

[94]  Applicant Reactions to Selection , 2016 .

[95]  J. Edwards,et al.  Using accumulated knowledge to calibrate theoretical propositions , 2014 .

[96]  D. Ones,et al.  Mechanical versus clinical data combination in selection and admissions decisions: a meta-analysis. , 2013, The Journal of applied psychology.

[97]  Colin Camerer.  Judgment and decision making, J. Frank Yates. Englewood Cliffs, New Jersey, Prentice-Hall inc. 1990 , 1991 .

[98]  Satoris S. Culbertson,et al.  Moving Forward Indirectly: Reanalyzing the Validity of Employment Interviews with Indirect Range Restriction Methodology , 2014 .

[99]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[100]  E. Brunswik.  Perception and the Representative Design of Psychological Experiments , 1957 .

[101]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[102]  G. Brier,et al.  External correspondence: Decompositions of the mean probability score , 1982 .

[103]  Stacey M. Whitecotton.  The effects of experience and a decision aid on the slope, scatter, and bias of earnings forecasts , 1996 .

[104]  J. Edwards,et al.  The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research , 2010 .

[105]  Robert L. Dipboye,et al.  Self-Fulfilling Prophecies in the Selection-Recruitment Interview , 1982 .

[106]  R. Nisbett,et al.  The dilution effect: Nondiagnostic information weakens the implications of diagnostic information , 1981, Cognitive Psychology.

[107]  Kenneth R. Hammond,et al.  Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice , 2000 .

[108]  J. Sauer,et al.  Effects of eye-closure on confidence-accuracy relations in eyewitness testimony , 2015 .

[109]  Jack B. Soll,et al.  Overconfidence: It Depends on How, What, and Whom You Ask. , 1999, Organizational behavior and human decision processes.

[110]  J. Frank Yates,et al.  Effects of procedural and outcome accountability on judgment quality , 1996 .

[111]  Robert D. Gatewood,et al.  Human Resource Selection , 1997 .