Assessing and improving the quality of security methodologies for distributed systems

Security methodologies represent systematic approaches for introducing security attributes into a system throughout the development lifecycle. While isolated attempts have been made to demonstrate the value of particular security methodologies, the “quality” of security methodologies, as such, has never been given due consideration; indeed, it has never been studied as a self‐standing topic. The literature therefore entirely lacks supportive artifacts that can provide a basis for assessing, and hence for improving, a security methodology's quality. In this paper, we fill the aforementioned gap by proposing a comprehensive quality framework and accompanying process, within the context of an existing approach to engineering security methodologies, which can be used for both (bottom‐up) quality assessment and (top‐down) quality improvement. The main framework elements can be extended and customized to allow an essentially arbitrary range of methodology features to be considered, thus forming a basis for flexible, fine‐grained quality control. We demonstrate the bottom‐up application of the latter framework and process on three real‐life security methodologies for distributed systems, taken as case studies. Based on the assessment results, we subsequently show in detail (for one) and briefly discuss (for the remaining set) how the case study methodologies can be re‐engineered to improve their quality.