Testing Security Requirements with Non-experts: Approaches and Empirical Investigations

Security testing has become a critical quality assurance technique to provide a sufficient degree of security. However, it is regarded to be too complex to be performed by system testers, who are non-experts in security. This paper provides two approaches to testing security requirements, one based on a Failure Modes, Vulnerabilities and Effect Analysis (FMVEA) and the other based on misuse cases, both suitable for testers who have domain knowledge but are not security experts. We perform a controlled experiment to empirically compare the two testing approaches based on the quality of the derived test cases. The results of the experiment show that the use of attack patterns in the misuse-case-based approach delivers test cases with a better alignment between requirements and security test cases as well as a higher amount of correct test cases.