Risk Based Testing and Metrics Risk Analysis Fundamentals and Metrics for software testing including a Financial Application case study

This paper provides an overview of risk analysis fundamentals, focusing on software testing with the key objectives of reducing the cost of the project test phase and reducing future potential production costs by optimising the test process. The phases of Risk Identification, Risk Strategy, Risk Assessment, Risk Mitigation (Reduction) and Risk Prediction are discussed. Of particular interest is the use of metrics to identify the probability and the consequences of individual risks (errors) if they occur, and to monitor test progress. The body of this paper contains a case study of the system test stage of a project to develop a very flexible retail banking application with complex test requirements. The project required a methodology that would identify functions in their system where the consequence of a fault would be most costly (either to the vendor or to the vendor’s customers) and also a technique to identify those functions with the highest probability of faults. A risk analysis was performed and the functions with the highest risk exposure, in terms of probability and cost, were identified. A risk based approach to testing was introduced, i.e. during testing resources would be focused in those areas representing the highest risk exposure. To support this approach, a well defined, but flexible, test organisation was developed. The test process was strengthened and well-defined control procedures were introduced. The level of test documentation produced prior to test execution was kept to a minimum and as a result, more responsibility was passed to the individual performing the test. To support this approach, progress tracking metrics were essential to show the actual progress made and to calculate the resources required to complete the test activities.