Semi-automatic Generation of Competency Self-assessments for Performance Appraisal

Competency self-assessment for Performance Appraisal is receiving increasing attention from both researchers and practitioners. Nevertheless, the accuracy and supposed legitimacy of this type of assessment is still an issue. In the context of an industrial use case, we aim to develop and validate a computer-based competency self-assessment technology able to import any type of competency document (for performance appraisal, training need identification, career guidance) and generate semi-automatically self-assessment items. Following the model of Appraisal Effectiveness designed by Levy and Williams, our goal was to build an effective tool meaning that several perspectives must be taken into account: psychometric, cognitive, psychological, political and the reaction’s perspective. In this paper, we will only focus on one specific psychometric property (interrater reliability between an employee and its supervisor). According to a specific rating process and format, our Cross Skill™ technology showed promising results related to interrater reliability in a use case with bank officers and their supervisor.

[1]  Robert P. Tett,et al.  Personality tests at the crossroads: A response to Morgeson, Campion, Dipboye, Hollenbeck, Murphy, and Schmitt (2007). , 2007 .

[2]  Allen I. Huffcutt,et al.  Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. , 1997 .

[3]  P. Levy,et al.  The Social Context of Performance Appraisal: A Review and Framework for the Future , 2004 .

[4]  W. Borman,et al.  An Applied Examination of the Computerized Adaptive Rating Scale for Assessing Performance , 2017 .

[5]  Richard M. Luecht.  An Introduction to Assessment Engineering for Automatic Item Generation , 2012 .

[6]  Alexandre Baudet,et al.  A Hybrid Engineering Process for Semi-automatic Item Generation , 2016, TEA.

[7]  Adrian Furnham,et al.  HR Professionals' Beliefs About, and Knowledge of, Assessment Techniques and Psychometric Tests , 2008 .

[8]  Andrea De Lucia,et al.  The Art and Science of Competency Models: Pinpointing Critical Success Factors in Organizations , 1999 .

[9]  F. Schmidt,et al.  Measurement Error Obfuscates Scientific Knowledge: Path to Cumulative Knowledge Requires Corrections for Unreliability and Psychometric Meta-Analyses , 2014, Industrial and Organizational Psychology.

[10]  Jonathan Winterton,et al.  Typology of knowledge, skills and competences : clarification of the concept and prototype , 2006 .

[11]  A. Furnham.  Response bias, social desirability and dissimulation , 1986 .

[12]  Lyle M. Spencer,et al.  Competence at Work: Models for Superior Performance , 1993 .

[13]  George M. Alliger,et al.  The Susceptibility of Overt and Covert Integrity Tests to Coaching and Faking , 1996 .

[14]  Francis J. Yammarino,et al.  DOES SELF‐OTHER AGREEMENT ON LEADERSHIP PERCEPTIONS MODERATE THE VALIDITY OF LEADERSHIP AND PERFORMANCE PREDICTIONS? , 1992 .

[15]  T. Kline.  Psychological Testing: A Practical Approach to Design and Evaluation , 2005 .

[16]  Brian J. Ruggeberg,et al.  DOING COMPETENCIES WELL: BEST PRACTICES IN COMPETENCY MODELING , 2011 .

[17]  R. Goffin,et al.  Faking and personality assessment in personnel selection: Advancing models of faking. , 2009 .