A study of software security problem disclosure, correction and patching processes

Quantitative analysis of software security problems plays an important role in understanding software security. Information on how and when software security problems are disclosed, exploited in the field, fixed by developers and patched by users, is often analysed from a calendar time perspective. This provides worst-case assessment and effort-to-fix information, but is not directly related to actual operational impact of the discovered problems. Given that security problems are a subset of the more general category of software problems, employing usage metrics typically found in classical software reliability engineering, such as inservice time, appears to be a reasonable approach for assessing security problems. The main goal of this thesis is to investigate operational software security problem disclosure, correction and patching processes through publicly available information, and through that improve our understanding of of the issues, as well as enable better process and defense planning and related decision making. One of the issues one runs into almost immediately when studying open software security data is the distributed nature and diversity of such data. The data reside in numerous data bases, in different formats, and it is a challenge to collect that information. The first step in the current work was to develop a set of tools for automated collection of linked information across public repositories. Investigated were products that follow a process of full disclosure of security problems before fixes are available (we call them “full disclosure” products), and those that disclose security problems along with fixes and possibly only limited information about them (we call them “limited disclosure” products). To analyse and understand collected information, a comprehensive security problem response model was developed that describes interactions of events associated with users, developers, attackers, software security problems, and fixes. The model captures the states through which a software may go based on the discovery, disclosure, exploit, failure, and correction of security problems. The model distinguishes itself from published models by emphasizing roles and operational impact perspectives. As part of the analyses, two sub-models are investigated for estimating the disclosure of unique security problems - the classical Logarithmic Poisson Execution Time (LPET) model, and a Bayesian model. The latter model was included to capture the subjective views of risk and exposure. Both models were found to work well - the LPET in the context of security problem rates across releases, and the Bayesian model in the context of disclosure of security problems per release. In combination with experimental data, the overall model was also used to investigate security problem disclosure, correction and patching policies. Time to discovery, time-to-disclosure, time-to-intrusion, time-to-patch-availability, and time-to-patch-application are some of the metrics in this context. Empirical results tell us that between 30% and 80% of the reported problems will fail in the field only if end-users interact with the attack mechanism (e.g., opening a malicious attachment in an email). We classify such problems as “voluntary” security problems. Early warning/disclosure of such problems may help users in taking precautions. An interesting question is “Under what conditions is the policy of early disclosure of voluntary security problems a good one?”. This is discussed from the perspectives where (a) users do not intervene in the installation of patches (we call it “automatic updates”) and (b) where users do intervene (we call it “non-automatic updates”). For a given set of values of the process metrics under consideration, it is shown what percentage of users should heed the warning for the policy of early disclosure to be effective. Several other such policies are examined and discussed.