The Battle for New York: A Case Study of Applied Digital Threat Modeling at the Enterprise Level

Digital security professionals use threat modeling to assess and improve the security posture of an organization or product. However, no threat-modeling techniques have been systematically evaluated in a real-world, enterprise environment. In this case study, we introduce formalized threat modeling to New York City Cyber Command: the primary digital defense organization for the most populous city in the United States. We find that threat modeling improved self-efficacy; 20 of 25 participants regularly incorporated it within their daily duties 30 days after training, without further prompting. After 120 days, implemented participantdesigned threat mitigation strategies provided tangible security benefits for NYC, including blocking 541 unique intrusion attempts, preventing the hijacking of five privileged user accounts, and addressing three public-facing server vulnerabilities. Overall, these results suggest that the introduction of threat modeling can provide valuable benefits in an enterprise setting.

[1]  H. Akaike.  A new look at the statistical model identification , 1974 .

[2]  Jane Cleland-Huang,et al.  How Well Do You Know Your Personae Non Gratae? , 2014, IEEE Softw..

[3]  Adam Shostack,et al.  Threat Modeling: Designing for Security , 2014 .

[4]  Tudor Dumitras,et al.  Vulnerability Disclosure in the Age of Social Media: Exploiting Twitter for Predicting Real-World Exploits , 2015, USENIX Security Symposium.

[5]  J. W. Atkinson.  Motivational determinants of risk-taking behavior. , 1957, Psychological review.

[6]  Andreas L. Opdahl,et al.  Eliciting security requirements with misuse cases , 2004, Requirements Engineering.

[7]  Fabio Massacci,et al.  An Experimental Comparison of Two Risk-Based Security Methods , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[8]  Dale C. Eikmeier.  Center of Gravity Analysis , 2004 .

[9]  Eric Michael Hutchins,et al.  Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains , 2010 .

[10]  John Mylopoulos,et al.  Modeling security requirements through ownership, permission and delegation , 2005, 13th IEEE International Conference on Requirements Engineering (RE'05).

[11]  Melanie C. Green,et al.  Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias , 2003 .

[12]  William E Gortney.  Department of Defense Dictionary of Military and Associated Terms , 2016 .

[13]  Donald Hedeker,et al.  Multilevel Models for Ordinal and Nominal Variables , 2008 .

[14]  David Lazer,et al.  Network Theory and Small Groups , 2004 .

[15]  Fabio Massacci,et al.  How to Select a Security Requirements Method? A Comparative Study with Students and Practitioners , 2012, NordSec.

[16]  Scott J. Shackelford.  Should Cybersecurity Be a Human Right? Exploring the ‘Shared Responsibility’ of Cyber Peace , 2017 .

[17]  Andreas L. Opdahl,et al.  Experimental comparison of attack trees and misuse cases for security threat identification , 2009, Inf. Softw. Technol..

[18]  Carl Colwill,et al.  Human factors in information security: The insider threat - Who can you trust these days? , 2009, Inf. Secur. Tech. Rep..

[19]  Bashar Nuseibeh,et al.  Security Requirements Engineering: A Framework for Representation and Analysis , 2008, IEEE Transactions on Software Engineering.

[20]  Jeffrey A. Mattson,et al.  Defense in Depth: Foundation for Secure and Resilient IT Enterprises , 2006 .

[21]  Ketil Stølen,et al.  Model-Driven Risk Analysis - The CORAS Approach , 2010 .

[22]  Naomi Miyake,et al.  Constructive Interaction and the Iterative Process of Understanding , 1986, Cogn. Sci..

[23]  Haralambos Mouratidis,et al.  Integrating Security and Systems Engineering: Towards the Modelling of Secure Information Systems , 2003, CAiSE.

[24]  Donelson R. Forsyth.  Self-Serving Bias , 2008 .

[25]  John Ingham,et al.  Why do people use information technology? A critical review of the technology acceptance model , 2003, Inf. Manag..

[26]  Bruce Schneier,et al.  Toward a secure system engineering methodolgy , 1998, NSPW '98.

[27]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[28]  Michael Muckin,et al.  A Threat-Driven Approach to Cyber Security Methodologies , Practices and Tools to Enable a Functionally Integrated Cyber Security Organization , 2015 .

[29]  M. Orne.  On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. , 1962 .

[30]  D. Kolb,et al.  Learning Styles and Learning Spaces: Enhancing Experiential Learning in Higher Education , 2005 .

[31]  A. Bandura.  Perceived Self-Efficacy in Cognitive Development and Functioning , 1993, Educational Psychologist.

[32]  R. Tourangeau,et al.  Sensitive questions in surveys. , 2007, Psychological bulletin.

[33]  Laura Johnson,et al.  How Many Interviews Are Enough? , 2006 .

[34]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[35]  Mario Piattini,et al.  Applying a Security Requirements Engineering Process , 2006, ESORICS.

[36]  Daniel L. Moody,et al.  The method evaluation model: a theoretical model for validating information systems design methods , 2003, ECIS.

[37]  A. Bandura.  GUIDE FOR CONSTRUCTING SELF-EFFICACY SCALES , 2006 .

[38]  Josiah A. B. S. Dykstra,et al.  Acting in the unknown: the cynefin framework for managing cybersecurity risk in dynamic decision making , 2016, 2016 International Conference on Cyber Conflict (CyCon U.S.).