Using Design-Science Based Gamification to Improve Organizational Security Training and Compliance

ABSTRACT We conducted a design-science research project to improve an organization’s compound problems of (1) unsuccessful employee phishing prevention and (2) poorly received internal security training. To do so, we created a gamified security training system focusing on two factors: (1) enhancing intrinsic motivation through gamification and (2) improving security learning and efficacy. Our key theoretical contribution is proposing a recontextualized kernel theory from the hedonic-motivation system adoption model that can be used to assess employee security constructs along with their intrinsic motivations and coping for learning and compliance. A six-month field study with 420 participants shows that fulfilling users’ motivations and coping needs through gamified security training can result in statistically significant positive behavioral changes. We also provide a novel empirical demonstration of the conceptual importance of “appropriate challenge” in this context. We vet our work using the principles of proof-of-concept and proof-of-value, and we conclude with a research agenda that leads toward final proof-in-use.

[1]  Tamara Dinev,et al.  Managing Employee Compliance with Information Security Policies: The Critical Role of Top Management and Organizational Culture , 2012, Decis. Sci..

[2]  J. Banfield,et al.  Increasing Student Intrinsic Motivation And Self-Efficacy Through Gamification Pedagogy , 2014 .

[3]  John E. Mathieu,et al.  INDIVIDUAL AND SITUATIONAL INFLUENCES ON THE DEVELOPMENT OF SELF‐EFFICACY: IMPLICATIONS FOR TRAINING EFFECTIVENESS , 2006 .

[4]  Tejaswini Herath,et al.  A review and analysis of deterrence theory in the IS security literature: making sense of the disparate findings , 2011, Eur. J. Inf. Syst..

[5]  Venkatesh,et al.  Computer Technology Training in the Workplace: A Longitudinal Investigation of the Effect of Mood. , 1999, Organizational behavior and human decision processes.

[6]  Andrea Back,et al.  Shadow it – A View from Behind the Curtain , 2014, Comput. Secur..

[7]  Paul Benjamin Lowry,et al.  Cognitive‐affective drivers of employees' daily compliance with information security policies: A multilevel, longitudinal study , 2019, Inf. Syst. J..

[8]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[9]  Joseph J. Martocchio,et al.  Relationship between conscientiousness and learning in employee training: mediating influences of self-deception and self-efficacy. , 1997, The Journal of applied psychology.

[10]  Tom L. Roberts,et al.  Insiders' Protection of Organizational Information Assets: Development of a Systematics-Based Taxonomy and Theory of Diversity for Protection-Motivated Behaviors , 2013, MIS Q..

[11]  L. Wood,et al.  From the Authors , 2003, European Respiratory Journal.

[12]  Merrill Warkentin,et al.  Leader’s dilemma game: An experimental design for cyber insider threat research , 2015, Information Systems Frontiers.

[13]  M. Csíkszentmihályi,et al.  Beyond Boredom and Anxiety: The Experience of Play in Work and Games. , 1977 .

[14]  Steven J. Pentland,et al.  A Video-Based Screening System for Automated Risk Assessment Using Nuanced Facial Features , 2017, J. Manag. Inf. Syst..

[15]  Dennis F. Galletta,et al.  It’s complicated: explaining the relationship between trust, distrust, and ambivalence in online transaction relationships using polynomial regression analysis and response surface analysis , 2017, Eur. J. Inf. Syst..

[16]  Lan Cao,et al.  Evolution of Governance: Achieving Ambidexterity in IT Outsourcing , 2013, J. Manag. Inf. Syst..

[17]  Paul Benjamin Lowry,et al.  security and privacy research lies , 2017 .

[18]  Mark J. Nelson,et al.  Soviet and American precursors to the gamification of work , 2012, MindTrek.

[19]  Stephanie C Payne,et al.  A meta-analytic examination of the goal orientation nomological net. , 2007, The Journal of applied psychology.

[20]  Qingguo Ma,et al.  Inverted U-Shaped Curvilinear Relationship between Challenge and One's Intrinsic Motivation: Evidence from Event-Related Potentials , 2017, Front. Neurosci..

[21]  A. Bandura Perceived Self-Efficacy in Cognitive Development and Functioning , 1993, Educational Psychologist.

[22]  Chuan-Hoo Tan,et al.  Enhancing User-Game Engagement Through Software Gaming Elements , 2014, J. Manag. Inf. Syst..

[23]  Darryl Charles,et al.  Toward an understanding of flow in video games , 2008, CIE.

[24]  G. Johns The Essential Impact of Context on Organizational Behavior , 2006 .

[25]  John W. Rice,et al.  The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[26]  Tom L. Roberts,et al.  The Impact of Organizational Commitment on Insiders’ Motivation to Protect Organizational Information Assets , 2015, J. Manag. Inf. Syst..

[27]  A. Kohn Why incentive plans cannot work , 1993 .

[28]  Scott Nicholson,et al.  A RECIPE for Meaningful Gamification , 2015 .

[29]  A. J. Ferguson Fostering E-Mail Security Awareness: The West Point Carronade , 2005 .

[30]  Paul Benjamin Lowry,et al.  Using IT Design to Prevent Cyberbullying , 2017, J. Manag. Inf. Syst..

[31]  Jingguo Wang,et al.  Coping Responses in Phishing Detection: An Investigation of Antecedents and Consequences , 2017, Inf. Syst. Res..

[32]  Dianne Cyr,et al.  Perceived interactivity leading to e-loyalty: Development of a model for cognitive-affective user responses , 2009, Int. J. Hum. Comput. Stud..

[33]  Paul Benjamin Lowry,et al.  Increasing Accountability Through User-Interface Design Artifacts: A New Approach to Addressing the Problem of Access-Policy Violations , 2015, MIS Q..

[34]  Ryan T. Wright,et al.  Training to Mitigate Phishing Attacks Using Mindfulness Techniques , 2017, J. Manag. Inf. Syst..

[35]  Daniel J. Veit,et al.  Gamification - A Novel Phenomenon or a New Wrapping for Existing Concepts? , 2015, ICIS.

[36]  Jay F. Nunamaker,et al.  Autonomous Scientifically Controlled Screening Systems for Detecting Information Purposely Concealed by Individuals , 2014, J. Manag. Inf. Syst..

[37]  Zi-Lin He,et al.  Thinking about U: Theorizing and testing U‐ and inverted U‐shaped relationships in strategy research , 2016 .

[38]  M. Csíkszentmihályi Finding Flow: The Psychology of Engagement with Everyday Life , 1997 .

[39]  Mikko T. Siponen,et al.  Neutralization: New Insights into the Problem of Employee Systems Security Policy Violations , 2010, MIS Q..

[40]  E. Deci,et al.  Quality of Learning With an Active Versus Passive Motivational Set , 1984 .

[41]  Gurpreet Dhillon,et al.  A Framework and Guidelines for Context-Specific Theorizing in Information Systems Research , 2014, Inf. Syst. Res..

[42]  Chi-Chang Chen,et al.  An online game approach for improving students' learning performance in web-based problem-solving activities , 2012, Comput. Educ..

[43]  Luis de Marcos,et al.  Gamifying learning experiences: Practical implications and outcomes , 2013, Comput. Educ..

[44]  Samir Chatterjee,et al.  A Design Science Research Methodology for Information Systems Research , 2008 .

[45]  I. Ajzen The theory of planned behavior , 1991 .

[46]  Lorrie Faith Cranor,et al.  Teaching Johnny not to fall for phish , 2010, TOIT.

[47]  Ryan J. Baxter,et al.  Applying Basic Gamification Techniques to IT Compliance Training: Evidence from the Lab and Field , 2015, J. Inf. Syst..

[48]  Jay F. Nunamaker,et al.  Detecting Fake Websites: The Contribution of Statistical Learning Theory , 2010, MIS Q..

[49]  Robin L. Wakefield,et al.  Mobile computing: a user study on hedonic/utilitarian mobile device usage , 2006, Eur. J. Inf. Syst..

[50]  Deborah Richards,et al.  VirSchool: The effect of background music and immersive display systems on memory for facts learned in an educational virtual environment , 2012, Comput. Educ..

[51]  Wynne W. Chin,et al.  A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic - Mail Emotion/Adoption Study , 2003, Inf. Syst. Res..

[52]  D. Brooks,et al.  Evidence for striatal dopamine release during a video game , 1998, Nature.

[53]  Paul Benjamin Lowry,et al.  Proposing the control‐reactance compliance model (CRCM) to explain opposing motivations to comply with organisational information security policies , 2015, Inf. Syst. J..

[54]  Nathan W. Twyman,et al.  Taking "Fun and Games" Seriously: Proposing the Hedonic-Motivation System Adoption Model (HMSAM) , 2012, J. Assoc. Inf. Syst..

[55]  Richard D. Johnson,et al.  Research Report: The Role of Behavioral Modeling in Computer Skills Acquisition: Toward Refinement of the Model , 2000, Inf. Syst. Res..

[56]  B. Frey,et al.  Motivation, Knowledge Transfer, and Organizational Forms , 2000 .

[57]  Kevin Ortbach,et al.  The inner and the outer model in explanatory design theory: the case of designing electronic feedback systems , 2016, Eur. J. Inf. Syst..

[58]  M. Csíkszentmihályi Beyond boredom and anxiety , 1975 .

[59]  Dazhong Wu,et al.  Factors That Influence Employees’ Security Policy Compliance: An Awareness-Motivation-Capability Perspective , 2018, J. Comput. Inf. Syst..

[60]  Tejaswini Herath,et al.  Encouraging information security behaviors in organizations: Role of penalties, pressures and perceived effectiveness , 2009, Decis. Support Syst..

[61]  R. W. Rogers,et al.  A meta-analysis of research on protection motivation theory. , 2000 .

[62]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[63]  D. Simons,et al.  The effects of video game playing on attention, memory, and executive control. , 2008, Acta psychologica.

[64]  Chung-Yuan Hsu,et al.  Facilitating Third Graders’ Acquisition of Scientific Concepts through Digital Game-Based Learning: The Effects of Self-Explanation Principles , 2012 .

[65]  Jay F. Nunamaker,et al.  Systems Development in Information Systems Research , 1990, J. Manag. Inf. Syst..

[66]  Paul Benjamin Lowry,et al.  Using Accountability to Reduce Access Policy Violations in Information Systems , 2013, J. Manag. Inf. Syst..

[67]  Y. Fried,et al.  Location, location, location: contextualizing organizational research* , 2001 .

[68]  Jingguo Wang,et al.  Overconfidence in Phishing Email Detection , 2016, J. Assoc. Inf. Syst..

[69]  S. Kühn,et al.  Playing Super Mario induces structural brain plasticity: gray matter changes resulting from training with a commercial video game , 2014, Molecular Psychiatry.

[70]  Merrill Warkentin,et al.  Beyond Deterrence: An Expanded View of Employee Computer Abuse , 2013, MIS Q..

[71]  Jan H. Kietzmann,et al.  Understanding Gamification of Consumer Experiences , 2014 .

[72]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[73]  Viswanath Venkatesh,et al.  Predicting Collaboration Technology Use: Integrating Technology Adoption and Collaboration Research , 2010, J. Manag. Inf. Syst..

[74]  Jay F. Nunamaker,et al.  The Last Research Mile: Achieving Both Rigor and Relevance in Information Systems Research , 2015, J. Manag. Inf. Syst..

[75]  Paul Benjamin Lowry,et al.  A Tale of Two Deterrents: Considering the Role of Absolute and Restrictive Deterrence to Inspire New Directions in Behavioral and Organizational Security Research , 2018, J. Assoc. Inf. Syst..

[76]  Radhika Santhanam,et al.  Toward Meaningful Engagement: A Framework for Design and Research of Gamified Information Systems , 2017, MIS Q..

[77]  Paul Benjamin Lowry,et al.  A New Approach to the Problem of Access Policy Violations: Increasing Perceptions of Accountability through the User Interface , 2015 .

[78]  Tom L. Roberts,et al.  Examining the Relationship of Organizational Insiders' Psychological Capital with Information Security Threat and Coping Appraisals , 2017, Comput. Hum. Behav..

[79]  Tuure Tuunanen,et al.  Extending Critical Success Factors Methodology to Facilitate Broadly Participative Information Systems Planning , 2003, J. Manag. Inf. Syst..

[80]  Paul Benjamin Lowry,et al.  Proposing the Multimotive Information Systems Continuance Model (MISC) to Better Explain End-User System Evaluations and Continuance Intentions , 2015, J. Assoc. Inf. Syst..

[81]  Jay F. Nunamaker,et al.  Creating High-Value Real-World Impact through Systematic Programs of Research , 2017, MIS Q..

[82]  Dennis F. Galletta,et al.  User Awareness of Security Countermeasures and Its Impact on Information Systems Misuse: A Deterrence Approach , 2009, Inf. Syst. Res..

[83]  Elena Karahanna,et al.  Time Flies When You're Having Fun: Cognitive Absorption and Beliefs About Information Technology Usage , 2000, MIS Q..

[84]  M. Adams,et al.  Cybersecurity Skills Training: An Attacker-Centric Gamified Approach , 2015 .

[85]  Li Chunlin,et al.  Optimal scheduling across public and private clouds in complex hybrid cloud environment , 2015, Information Systems Frontiers.

[86]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[87]  Sebastian Deterding,et al.  Gamification: designing for motivation , 2012, INTR.

[88]  Jay F. Nunamaker,et al.  Toward a broader vision for Information Systems , 2011, TMIS.

[89]  David J. Shernoff,et al.  Student engagement as a function of environmental complexity in high school classrooms , 2016 .

[90]  Kuang-Wei Wen,et al.  Organizations' Information Security Policy Compliance: Stick or Carrot Approach? , 2012, J. Manag. Inf. Syst..

[91]  Tom L. Roberts,et al.  Leveraging fairness and reactance theories to deter reactive computer abuse following enhanced organisational information security policies: an empirical study of the influence of counterfactual reasoning and organisational trust , 2015, Inf. Syst. J..

[92]  LowryPaul Benjamin,et al.  Proposing the control-reactance compliance model CRCM to explain opposing motivations to comply with organisational information security policies , 2015 .

[93]  Merrill Warkentin,et al.  An Enhanced Fear Appeal Rhetorical Framework: Leveraging Threats to the Human Asset Through Sanctioning Rhetoric , 2015, MIS Q..

[94]  Paul Benjamin Lowry,et al.  The Role of Extra-Role Behaviors and Social Controls in Information Security Policy Effectiveness , 2015, Inf. Syst. Res..

[95]  Qing Hu,et al.  Future directions for behavioral information security research , 2013, Comput. Secur..

[96]  I. Ajzen,et al.  Understanding Attitudes and Predicting Social Behavior , 1980 .

[97]  Dennis F. Galletta,et al.  What Do Systems Users Have to Fear? Using Fear Appeals to Engender Threats and Fear that Motivate Protective Security Behaviors , 2015, MIS Q..

[98]  Qingguo Ma,et al.  Close games versus blowouts: Optimal challenge reinforces one's intrinsic motivation to win. , 2016, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[99]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[100]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[101]  Angelos D. Keromytis,et al.  A market-based bandwidth charging framework , 2010, TOIT.

[102]  Izak Benbasat,et al.  Information Security Policy Compliance: An Empirical Study of Rationality-Based Beliefs and Information Security Awareness , 2010, MIS Q..

[103]  Chandrasekar Subramaniam,et al.  Determinants of the Choice of Open Source Software License , 2008, J. Manag. Inf. Syst..

[104]  David A. Edwards,et al.  Intercollegiate soccer: Saliva cortisol and testosterone are elevated during competition, and testosterone is related to status and social connectedness with teammates , 2006, Physiology & Behavior.

[105]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[106]  LowryPaul Benjamin,et al.  Leveraging fairness and reactance theories to deter reactive computer abuse following enhanced organisational information security policies , 2015 .