The importance of prewhitening in change point analysis under persistence

The presence of serial correlation in hydro-meteorological time series often makes the detection of deterministic gradual or abrupt changes with tests such as Mann–Kendall (MK) and Pettitt problematic. In this study we investigate the adverse impact of serial correlation on change point analyses performed by the Pettitt test. Building on methods developed for the MK test, different prewhitening procedures devised to remove the serial correlation are examined, and the effects of the sample size and strength of serial dependence on their performance are tested by Monte Carlo experiments involving the first-order autoregressive [AR(1)] process, fractional Gaussian noise (fGn), and fractionally integrated autoregressive [ARFIMA(1,d,0)] model. Results show that (1) the serial correlation affects the Pettitt test more than tests for slowly varying monotonic trends such as the MK test both for short-range and long-range persistence; (2) the most efficient prewhitening procedure based on AR(1) involves the simultaneous estimation of step change and lag-1 autocorrelation ρ, and bias correction of ρ estimates; (3) as expected, the effectiveness of the prewhitening procedure strongly depends upon the model selected to remove the serial correlation; (4) prewhitening procedures allow for a better control of the type I error resulting in rejection rates reasonably close to the nominal values. As ancillary results, (5) we show the ineffectiveness of the original formulation of the so-called trend-free prewhitening (TFPW) method and provide analytical results supporting a corrected version called TFPWcu; and (6) we propose an improved two-stage bias correction of ρ estimates for AR(1) signals.