Resilience Assessment Based on Performance Testing

Performance degradation and/or irregularity are often indicators of system instability. By applying the principle that average performance measures vary little under constant load in a stable system without periodic behavior, we can use performance metrics to anticipate instability within the system. We describe how to use that information to isolate the cause of observed instability and how to structure load tests to identify scenarios in which system instability is likely to occur. We discuss resilience assessment based on performance measurements. Specifically, we present our experience generating load tests and analyzing the performance testing results. We discuss how we have used these results for security, reliability and performance assessment. We discuss the conditions required for system stability and identify some of the causes for system instability, such as security attacks, quality problems, and queuing for system resources. We present a metric that can be used to assess some dimensions of system security, reliability and performance using data obtained from the execution of performance testing. In addition, we present the associated testing activities that are required to collect data for the required modeling and analysis activities, and to help track system security, reliability and performance. We illustrate the presented methodology with empirical results obtained by testing for security, reliability, and performance.