Resampling and extreme value statistics in air quality model performance evaluation

Abstract Specific data analysis techniques that will reveal the performance of air quality models in simulating the measured concentrations' cumulative distribution are discussed in this paper. The paper presents two types of analysis to compare model predictions with the measurements. In one analysis, extreme value statistics and the fitting of tail exponential distributions to both measured and predicted values are used in various ways to see if the measured and predicted values fit such distributions and to what degree the higher values of the cumulative frequency distributions coincide. In the second analysis, a resampling (bootstrap) technique is used to develop non-parametric confidence intervals for the entire cumulative distribution of the measured concentrations, and to derive empirical distributions for central tendency statistics and for differences between measured and predicted mean and median values. The analysis is focused so as to show 1. (1) why the resampling is necessary and the degree to which mistaken judgements can be made with and without the technique and 2. (2) comparisons between the discriminating capabilities of ‘tail fit’ type model evaluation and one using the resampling technique. It is shown that both bootstrap and extreme value statistics are needed to quantify the uncertainty associated with the model predictions. These techniques are applied to the model predictions from RAM and measurements from the Regional Air Pollution Study data base to demonstrate their utility in the quantitative assessment of the model performance.