The Forgotten Factor: Facts on Performance Evaluation and Its Dependence on Workloads

The performance of a computer system depends not only on its design and implementation, but also on the workloads it has to handle. Indeed, in some cases the workload can sway performance evaluation results. It is therefore crucially important that representative workloads be used for performance evaluation. This can be done by analyzing and modeling existing workloads. However, as more sophisticated workload models become necessary, there is an increasing need for the collection of more detailed data about workloads. This has to be done with an eye for those features that are really important.