The question “is this system secure?” is notoriously difficult to answer. The question implies that there is a system-wide property called “security,” which we can measure with some meaningful threshold of sufficiency. In this concept paper, we discuss the difficulty of measuring security sufficiency, either directly or through proxy such as the number of known vulnerabilities. We propose that the question can be better addressed by measuring confidence and risk in the decisions that depend on security. A novelty of this approach is that it integrates use of both subjective information (e.g. expert judgment) and empirical data. We investigate how this approach uses well-known methods from the discipline of decision-making under uncertainty to provide a more rigorous and useable measure of security sufficiency.
[1]
J. Berger.
Statistical Decision Theory and Bayesian Analysis
,
1988
.
[2]
Nancy R. Mead,et al.
Software Assurance Measurement -- State of the Practice
,
2013
.
[3]
Daniel Port,et al.
Empirical and face validity of software maintenance defect models used at the jet propulsion laboratory
,
2014,
ESEM '14.
[4]
R. Rosenkrantz.
Confidence Intervals vs Bayesian Intervals (1976)
,
1989
.
[5]
Barry Boehm,et al.
The Value Proposition for Assurance of JPL Systems
,
2014
.
[6]
Richard Kissel,et al.
Glossary of Key Information Security Terms
,
2014
.
[7]
Christopher J. Alberts,et al.
Risk-Based Measurement and Analysis: Application to Software Security
,
2012
.