Learning from Early Attempts to Measure Information Security Performance

The rapid evolution of threat ecosystems and the shifting focus of adversarial actions complicate efforts to assure security of an organization's computer networks. Efforts to build a rigorous science of security, one consisting of sound and reproducible empirical evaluations, start with measures of these threats, their impacts, and the factors that influence both attackers and victims. In this study, we present a careful examination of the issue of account compromise at two large academic institutions. In particular, we evaluate different hypotheses that capture common perceptions about factors influencing victims (e.g., demographics, location, behavior) and about the effectiveness of mitigation efforts (e.g., policy, education). While we present specific and sometimes surprising results of this analysis at our institutions, our goal is to highlight the need for similar in-depth studies elsewhere.