Metrics for Measuring the Efficacy of Critical-Infrastructure-Centric Cybersecurity Information Sharing Efforts

Efforts to secure and defend public- and private-sector cyber systems rely in part on information sharing. Information sharing strengthens the nation’s cybersecurity posture by allowing participating entities to have the broadest possible understanding of the tactics, techniques, and procedures of cyber threat actors and the vulnerabilities of cyber systems. Armed with this understanding, cyber defenders can better deter, prevent, disrupt, and recover from malicious cyber activity. Cybersecurity information sharing occurs in various fora in the public and private sectors. Within the Department of Homeland Security, the Office of Cybersecurity and Communications (CSC and 2) having the desired impact. This paper presents the suite of metrics and associated findings of the research, including its theoretical foundations. Guided by first principles and literature on information, information theory, decision theory, and uncertainty (as well as best practices in performance measurement), the paper recommends using a suite of metrics that measure various relevant inputs, processes, outputs, and outcomes of critical-infrastructure-centric cybersecurity information sharing efforts.

[1]  Eric Goldstein,et al.  An analysis of the primary authorities supporting and governing the efforts of the Department of Homeland Security to secure the cyberspace of the United States - final report. , 2011 .

[2]  Brian W. Cashell The Economic Impact of Cyber-Attacks , 2004 .

[3]  Neil A. Weiss,et al.  Introductory Statistics , 1982 .

[4]  Robert M. Gray,et al.  Entropy and Information , 1990 .

[5]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[6]  L. Lampkin Key Steps in Outcome Management , 2003 .

[7]  Lawrence B. Mohr Impact analysis for program evaluation , 1988 .

[8]  Harry P. Hatry Performance Measurement: Getting Results, Second Edition , 2007 .

[9]  Andrew Jaquith Security Metrics: Replacing Fear, Uncertainty, and Doubt , 2007 .

[10]  L. Floridi Information: A Very Short Introduction , 2010 .

[11]  J. Taylor An Introduction to Error Analysis , 1982 .

[12]  Luciano Floridi,et al.  What is the Philosophy of Information , 2002 .

[13]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[14]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[15]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[16]  Mark H. Moore,et al.  The Public Value Scorecard: A Rejoinder and an Alternative to 'Strategic Performance Measurement and Management in Non-Profit Organizations' by Robert Kaplan , 2003 .

[17]  Mark W. Lipsey,et al.  Evaluation: A Systematic Approach , 1979 .

[18]  Robert S. Kaplan,et al.  Conceptual Foundations of the Balanced Scorecard , 2010 .

[19]  Harry D. Raduege,et al.  Securing Cyberspace for the 44th Presidency , 2008 .

[20]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[21]  I. Higginson,et al.  Experimental and quasi-experimental designs , 2011 .

[22]  Matthew H. Fleming Issues in measuring the efficacy of a suspicious activity reports (SARs) regime , 2011 .