A descriptive study of Microsoft’s threat modeling technique

Microsoft’s STRIDE is a popular threat modeling technique commonly used to discover the security weaknesses of a software system. In turn, discovered weaknesses are a major driver for incepting security requirements. Despite its successful adoption, to date no empirical study has been carried out to quantify the cost and effectiveness of STRIDE. The contribution of this paper is the evaluation of STRIDE via a descriptive study that involved 57 students in their last master year in computer science. The study addresses three research questions. First, it assesses how many valid threats per hour are produced on average. Second, it evaluates the correctness of the analysis results by looking at the average number of false positives, i.e., the incorrect threats. Finally, it determines the completeness of the analysis results by looking at the average number of false negatives, i.e., the overlooked threats.

[1]  Daniel Mellado,et al.  A systematic review of security requirements engineering , 2010, Comput. Stand. Interfaces.

[2]  Ketil Stølen,et al.  Reducing the Effort to Comprehend Risk Models: Text Labels Are Often Preferred Over Graphical Means , 2011, Risk analysis : an official publication of the Society for Risk Analysis.

[3]  Gary McGraw,et al.  Software Security: Building Security In , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[4]  Paris Avgeriou,et al.  Relating Software Requirements and Architectures , 2011 .

[5]  William Yurcik,et al.  Threat Modeling as a Basis for Security Requirements , 2005 .

[6]  Michael Howard,et al.  The security development lifecycle : SDL, a process for developing demonstrably more secure software , 2006 .

[7]  Susan Elliott Sim,et al.  A Comparative Evaluation of Three Approaches to Specifying Security Requirements , 2006 .

[8]  Pierluigi Roberti,et al.  STS-tool: Socio-technical Security Requirements through social commitments , 2012, 2012 20th IEEE International Requirements Engineering Conference (RE).

[9]  Peter Torr,et al.  Demystifying the threat modeling process , 2005, IEEE Security & Privacy Magazine.

[10]  Ketil Stølen,et al.  On the comprehension of security risk scenarios , 2005, 13th International Workshop on Program Comprehension (IWPC'05).

[11]  Claes Wohlin,et al.  Using Students as Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment , 2000, Empirical Software Engineering.

[12]  Per Runeson,et al.  Using Students as Experiment Subjects – An Analysis on Graduate and Freshmen Student Data , 2003 .

[13]  Andreas L. Opdahl,et al.  Experimental comparison of attack trees and misuse cases for security threat identification , 2009, Inf. Softw. Technol..

[14]  Thomas Heyman,et al.  The Security Twin Peaks , 2011, ESSoS.

[15]  Wouter Joosen,et al.  Architectural design of a digital publishing system , 2006 .

[16]  Maritta Heisel,et al.  A comparison of security requirements engineering methods , 2010, Requirements Engineering.

[17]  Adam Shostack,et al.  Experiences Threat Modeling at Microsoft , 2008, MODSEC@MoDELS.

[18]  Walter F. Tichy,et al.  Hints for Reviewing Empirical Work in Software Engineering , 2000, Empirical Software Engineering.

[19]  Guttorm Sindre,et al.  Comparing Misuse Case and Mal-Activity Diagrams for Modelling Social Engineering Attacks , 2012, Int. J. Secur. Softw. Eng..

[20]  Maritta Heisel,et al.  A Pattern System for Security Requirements Engineering , 2007, The Second International Conference on Availability, Reliability and Security (ARES'07).

[21]  Axel van Lamsweerde,et al.  Elaborating security requirements by construction of intentional anti-models , 2004, Proceedings. 26th International Conference on Software Engineering.

[22]  John Mylopoulos,et al.  Security Requirements Engineering: The SI* Modeling Language and the Secure Tropos Methodology , 2010, Advances in Intelligent Information Systems.

[23]  Ketil Stølen,et al.  A graphical approach to risk identification, motivated by empirical investigations , 2006, MoDELS'06.

[24]  K. Schulz,et al.  Descriptive studies: what they can and cannot do , 2002, The Lancet.

[25]  Jeffrey C. Carver,et al.  A checklist for integrating student empirical studies with research and teaching goals , 2010, Empirical Software Engineering.

[26]  Michael N. Johnstone.  Threat Modelling with Stride and UML , 2010 .

[27]  Robert J. Ellison,et al.  Attack Trees , 2009, Encyclopedia of Biometrics.

[28]  Patrik Berander,et al.  Using students as subjects in requirements prioritization , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[29]  Inger Anne Tøndel,et al.  Idea: Reusability of Threat Models - Two Approaches with an Experimental Evaluation , 2010, ESSoS.

[30]  Bruce Potter.  Threat Modelling: Microsoft SDL Threat Modelling Tool , 2009 .

[31]  Andreas L. Opdahl,et al.  Eliciting security requirements with misuse cases , 2004, Requirements Engineering.

[32]  Andreas L. Opdahl,et al.  Experimental Comparison of Misuse Case Maps with Misuse Cases and System Architecture Diagrams for Eliciting Security Vulnerabilities and Mitigations , 2011, 2011 Sixth International Conference on Availability, Reliability and Security.

[33]  Danny Dhillon,et al.  Developer-Driven Threat Modeling: Lessons Learned in the Trenches , 2011, IEEE Security & Privacy.

[34]  Rick Kazman,et al.  Evaluating Software Architectures: Methods and Case Studies , 2001 .

[35]  Jeffrey A. Ingalsbe,et al.  Threat Modeling: Diving into the Deep End , 2008, IEEE Software.

[36]  Andreas L. Opdahl,et al.  Templates for Misuse Case Description , 2001 .

[37]  Haralambos Mouratidis,et al.  Secure Tropos: a Security-Oriented Extension of the Tropos Methodology , 2007, Int. J. Softw. Eng. Knowl. Eng..

[38]  Bashar Nuseibeh,et al.  Security Requirements Engineering: A Framework for Representation and Analysis , 2008, IEEE Transactions on Software Engineering.

[39]  Bashar Nuseibeh,et al.  Weaving Together Requirements and Architectures , 2001, Computer.

[40]  Wouter Joosen,et al.  A privacy threat analysis framework: supporting the elicitation and fulfillment of privacy requirements , 2011, Requirements Engineering.

[41]  Andreas Schaad,et al.  TAM2: automated threat analysis , 2012, SAC '12.

[42]  Claes Wohlin,et al.  Using students as subjects - an empirical evaluation , 2008, ESEM '08.