Gamified or traditional situational judgement test? A moderated mediation model of recommendation intentions via organizational attractiveness

ABSTRACT Recently, gamification and gamified assessment methods have attracted increasing attention among both scholars and practitioners in recruitment and selection. However, little is known about the role of gamified assessment methods in determining applicant outcomes and, even more, their effect compared to more traditional methods. Using signalling theory as an overarching theoretical framework, the present study attempts to contribute to this gap by comparing the effect of a gamified situational judgement test which includes game elements, such as avatars, narrative and fantasy on applicants’ recommendation intentions through their organizational attractiveness compared to a traditional situational judgement test. Moreover, we suggest that these effects are moderated by applicants’ video gaming experience. Using experimental design, we found that the positive effect of the gamified assessment method compared to the traditional method on organizational attractiveness and, in turn, recommendation intentions was significant only for those with a high level of video gaming experience. Implications for theory and practice are discussed.

[1]  P. Levy,et al.  Organizational Web Sites: Web Site Content and Style as Determinants of Organizational Attraction , 2003 .

[2]  Klaus G. Melchers,et al.  What Do Consistency and Personableness in the Interview Signal to Applicants? Investigating Indirect Effects on Organizational Attractiveness Through Symbolic Organizational Attributes , 2018, Journal of Business and Psychology.

[3]  Richard N. Landers,et al.  Gamifying a situational judgment test with immersion and control game elements , 2020 .

[4]  D. Mohr,et al.  Personality trait inferences about organizations: development of a measure and assessment of construct validity. , 2004, The Journal of applied psychology.

[5]  Larry E. Toothaker,et al.  Multiple Regression: Testing and Interpreting Interactions , 1991 .

[6]  B. Schreurs,et al.  The Relation between Selection Expectations, Perceptions and Organizational Attraction: A Test of Competing Models , 2010 .

[7]  Udo Konradt,et al.  Fairness Perceptions in Web‐Based Selection: Impact on Applicants’ Pursuit Intentions, Recommendation Intentions, and Intentions to Reapply , 2013 .

[8]  Filip Lievens,et al.  The relation of instrumental and symbolic attributes to a company's attractiveness as an employer. , 2003 .

[9]  Richard N. Landers,et al.  Enhancing instructional outcomes with gamification: An empirical test of the Technology-Enhanced Training Effectiveness Model , 2017, Comput. Hum. Behav..

[10]  Howard E. Miller,et al.  Recruiter and Job Influences on Candidates for Employment , 1983 .

[11]  Daniel B. Horn,et al.  The roles of task difficulty and prior videogame experience on performance and motivation in instructional videogames , 2008, Comput. Hum. Behav..

[12]  M. A. Campion,et al.  APPLICANT REACTIONS TO SELECTION: DEVELOPMENT OF THE SELECTION PROCEDURAL JUSTICE SCALE (SPJS) , 2001 .

[13]  Andrew B. Collmus,et al.  Game-Thinking Within Social Media to Recruit and Select Job Candidates , 2016 .

[14]  John P. Hausknecht,et al.  Applicant Reactions to Selection Procedures: An Updated Model and Meta-Analysis , 2004 .

[15]  Konstantina Georgiou,et al.  Gamification in Employee Selection: The Development of a Gamified Assessment , 2019, International Journal of Selection and Assessment.

[16]  G. V. Hoye.  Recruitment sources and organizational attraction: A field study of Belgian nurses , 2012 .

[17]  Tammy Schellens,et al.  Students' perceptions about the use of video games in the classroom , 2010, Comput. Educ..

[18]  Sven Laumer,et al.  Online Gaming to Find a New Job – Examining Job Seekers' Intention to Use Serious Games as a Self-Assessment Tool , 2012 .

[19]  Jeffrey R. Edwards,et al.  Person-job fit:: A conceptual integration, literature review, and methodological critique. , 1991 .

[20]  Greet van Hoye.  Recruitment sources and organizational attraction: A field study of Belgian nurses , 2012 .

[21]  I. Nikolaou.  Social Networking Web Sites in Job Search and Employee Recruitment , 2014 .

[22]  Richard N. Landers,et al.  Correcting Misconceptions About Gamification of Assessment: More Than SJTs and Badges , 2016, Industrial and Organizational Psychology.

[23]  Kelly A. Piasentin,et al.  Applicant attraction to organizations and job choice: a meta-analytic review of the correlates of recruiting outcomes. , 2005, The Journal of applied psychology.

[24]  Erica N. Drew,et al.  Candidate Reactions to Simulations and Media-Rich Assessments in Personnel Selection , 2013 .

[25]  M. Taylor,et al.  Organizational Recruitment: Enhancing the Intersection of Research and Practice , 2000 .

[26]  N LandersRichard.  Developing a Theory of Gamified Learning , 2014 .

[27]  Amy L. Kristof.  PERSON-ORGANIZATION FIT: AN INTEGRATIVE REVIEW OF ITS CONCEPTUALIZATIONS, MEASUREMENT, AND IMPLICATIONS , 1996 .

[28]  Ann Marie Ryan,et al.  The Unrealized Potential of Technology in Selection Assessment , 2019, Revista de Psicología del Trabajo y de las Organizaciones.

[29]  G. V. Hoye.  Nursing recruitment: relationship between perceived employer image and nursing employees’ recommendations , 2008 .

[30]  R. Landers.  Developing a Theory of Gamified Learning , 2014 .

[31]  James W. Smither,et al.  APPLICANT REACTIONS TO SELECTION PROCEDURES , 2006 .

[32]  Richard N. Landers,et al.  Gamifying Recruitment, Selection, Training, and Performance Management: Game-Thinking in Human Resource Management , 2015 .

[33]  Konstantina Georgiou,et al.  Are applicants in favor of traditional or gamified assessment methods? Exploring applicant reactions towards a gamified selection method , 2020, Comput. Hum. Behav..

[34]  F. Lievens,et al.  Video-based versus written situational judgment tests: a comparison in terms of predictive validity. , 2006, The Journal of applied psychology.

[35]  Robert E. Ployhart,et al.  Low Fidelity Simulations , 2015 .

[36]  F. Lievens,et al.  Social Influences in Recruitment: When is Word‐of‐Mouth Most Effective? , 2016 .

[37]  J. Oostrom,et al.  Employee Recruitment, Selection, and Assessment : Contemporary Issues for Theory and Practice , 2015 .

[38]  A. Mavletova.  Web Surveys Among Children and Adolescents , 2015 .

[39]  Aryeh Lazar,et al.  The influence of prefeedback selection justice on perceptions of overall procedural justice and organizational attractiveness in a real-life selection procedure , 2007 .

[40]  S. Carless.  Person-job fit versus person-organization fit as predictors of organizational attraction and job acceptance intentions : A longitudinal study , 2005 .

[41]  D. Gomes,et al.  Organizational attractiveness and prospective applicants' intentions to apply , 2011 .

[42]  David M. Saunders,et al.  Effects of application blanks and employment equity on applicant reactions and job pursuit intentions , 1995 .

[43]  Eduardo Salas,et al.  Toward a Taxonomy Linking Game Attributes to Learning , 2012 .

[44]  Chelsea Willness,et al.  Why Are Job Seekers Attracted by Corporate Social Performance? Experimental and Field Tests of Three Signal-Based Mechanisms , 2014 .

[45]  Paul E. Spector.  Do Not Cross Me: Optimizing the Use of Cross-Sectional Designs , 2019, Journal of Business and Psychology.

[46]  Filip Lievens,et al.  Tapping the grapevine: a closer look at word-of-mouth as a recruitment source. , 2009, The Journal of applied psychology.

[47]  Daniel M. Cable,et al.  Establishing the dimensions, sources and value of job seekers' employer knowledge during recruitment , 2001 .

[48]  J. Breaugh.  Employee recruitment. , 2013, Annual review of psychology.

[49]  C. Stevens,et al.  The relationship between early recruitment-related activities and the application decisions of new labor-market entrants: a brand equity approach to recruitment. , 2002, The Journal of applied psychology.

[50]  Harold W. Goldstein,et al.  Book Review:The Wiley Blackwell Handbook of the Psychology of Recruitment, Selection and Employee Retention , 2017, Assessment and Development Matters.

[51]  Derek Chapman,et al.  Gamifying the employee recruitment process , 2013, Gamification.

[52]  David R. Michael,et al.  Serious Games: Games That Educate, Train, and Inform , 2005 .

[53]  Selection fairness information and applicant reactions: a longitudinal field study. , 2002 .

[54]  Neil Anderson,et al.  Applicant perspectives on employee selection systems , 2017 .

[55]  Michael S. Fetzer,et al.  Gamification, Serious Games and Personnel Selection , 2017 .

[56]  Rachel C. Callan,et al.  How to Avoid the Dark Side of Gamification: Ten Business Scenarios and Their Unintended Consequences , 2015 .

[57]  I. Nikolaou,et al.  Fairness Reactions to the Employment Interview , 2018, Revista de Psicología del Trabajo y de las Organizaciones.

[58]  Deborah K. Reed,et al.  Students’ Perceptions of a Gamified Reading Assessment , 2020, Journal of Special Education Technology.

[59]  Andrew F. Hayes,et al.  The Analysis of Mechanisms and Their Contingencies: PROCESS versus Structural Equation Modeling , 2017 .

[60]  Kimberly M. Lukaszewski,et al.  Factors affecting the effectiveness and acceptance of electronic selection systems , 2013 .

[61]  C. Stevens,et al.  The relationship between early recruitment-related activities and the application decisions of new labor-market entrants: a brand equity approach to recruitment. , 2002 .

[62]  Stephen A. Woods,et al.  Personnel selection in the digital age: a review of validity and applicant reactions, and future research challenges , 2019, European Journal of Work and Organizational Psychology.

[63]  S. Gilliland,et al.  Effects of procedural and distributive justice on reactions to a selection system. , 1994 .

[64]  William A. Sodeman,et al.  Millennials and Technology: Addressing the Communication Gap in Education and Practice , 2014 .

[65]  Maire Kerrin,et al.  Evaluations of situational judgement tests to assess non‐academic attributes in selection , 2012, Medical education.

[66]  Kenton O'Hara,et al.  Gamification. using game-design elements in non-gaming contexts , 2011, CHI Extended Abstracts.

[67]  Jian Tang,et al.  Points, stories, worlds, and diegesis: Comparing player experiences in two citizen science games , 2015, Comput. Hum. Behav..

[68]  A. Kristof-brown,et al.  CONSEQUENCES OF INDIVIDUALS' FIT AT WORK: A META-ANALYSIS OF PERSON-JOB, PERSON-ORGANIZATION, PERSON-GROUP, AND PERSON-SUPERVISOR FIT , 2005 .

[69]  N. Schmitt,et al.  Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions. , 1997, The Journal of applied psychology.

[70]  Filip Lievens,et al.  Measuring Attraction to Organizations , 2003 .

[71]  Konstantina Georgiou,et al.  Exploring the Relationship of a Gamified Assessment with Performance , 2019, The Spanish Journal of Psychology.

[72]  Parbudyal Singh,et al.  Signaling theory and applicant attraction outcomes , 2011 .

[73]  M. Spence.  Job Market Signaling , 1973 .

[74]  S. Rynes,et al.  The Importance of Recruitment in Job Choice: A Different Way of Looking , 2006 .

[75]  Frederik Anseel,et al.  The Mediating Role of Feedback Acceptance in the Relationship between Feedback and Attitudinal and Performance Outcomes , 2009 .

[76]  J. Ziegert,et al.  Why Are Individuals Attracted to Organizations? , 2005 .

[77]  A. Hayes.  Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach , 2013 .

[78]  Thomas E. Becker.  Potential Problems in the Statistical Control of Variables in Organizational Research: A Qualitative Analysis With Recommendations , 2005 .

[79]  Benjamin Schneider,et al.  The ASA framework: An update. , 1995 .

[80]  T. Guin,et al.  Myths and Realities of Respondent Engagement in Online Surveys , 2012 .

[81]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .