Sensitivity of reliability growth models to operational profile errors

The estimation of the operational profile is one of the key factors during the use of software reliability growth models. However, the operational profile can be very difficult to estimate in particular applications such as software used for process control. In other cases, a single operational profile may not be sufficient to describe the use of the product by a number of different customers. An operational profile may also change during the development of software or during its operational usage. All these cases may lead to errors in the estimation of the operational profile. The paper describes an empirical evaluation of the sensitivity of reliability growth models to errors in the estimation of the operational profiles. Some reliability growth models are applied during the testing phase of a software system. The particular characteristics of the case study allow the measurement of the actual reliability growth of the software and its comparison with the estimations provided by the models. Measurement and comparison are repeated for different operational profiles giving information about the effect of a possible error in the estimation of the operational profile. Results show that errors in the operational profile estimation do not heavily affect reliability estimates and that their influence is strongly dependent on the accuracy with which the software system has been tested.