An Extended Systematic Literature Review on Classification , Structuring and Assessment of Evidence for Safety Compliance

Context: Critical systems as found in domains such as avionics, railways, and automotive are often subject to a formal process of safety assessment or certification. The goal of this process is to ensure that these systems will operate safely without posing undue risks to the user, the public, or the environment. Safety is usually assured by compliance with safety standards, which involves providing evidence to show that the safety criteria of the standards are met. Oftentimes, standards are ambiguous and extremely time consuming to understand, due to its sheer size and the fact that textual standards are prone to subjective interpretation. Objective: As a prerequisite to comply with standards, safety professionals need knowledge of how to classify different types of evidence, how to structure the evidence, and how to assess them. This paper is aimed towards developing such a body of knowledge. Method: This paper reports a Systematic Literature Review (SLR) of 216 papers published between 1990-2012, obtained after a multi-stage selection process. The SLR identifies and classifies the information and artefacts considered as evidence for safety, analyses the existing techniques for evidence structuring and assessment. Results: The results of the review were used to build an evidence taxonomy which will be particularly relevant to practitioners seeking a better understand about evidence requirements as well as to researchers conducting research in this area. The taxonomy has 49 different evidence types that were collected from the 216 papers. The paper also reports on most identified techniques for evidence structuring and assessment. In addition, the SLR summarizes the challenges noted in the literature in relation to provision of safety evidence and analyses commonalities between different application domains. The paper further discusses the implications of the results of the literature review for future research in the application domains identified in the review. Conclusion: The paper, to our knowledge, is the first systematic review concerning evidence for safety. The results provide useful insights for both research and practice. Particularly, the evidence classification developed provides a concrete basis for learning about and tailoring the various types of evidence that practitioners need to provide in support of safety. As a major finding in the review, the results suggest that there is a strong need for more practice-oriented and industry-driven empirical studies in this area.