Cleaning high-frequency velocity profile data with autoregressive moving average (ARMA) models

Abstract High resolution velocity profiling instruments have enabled a new generation of turbulence studies by greatly increasing the amount and quality of simultaneous velocity measurements that can be obtained. As with all velocity profiling instruments, however, the collected data are susceptible to erroneous spikes and poor quality time series that must be corrected or removed prior to analysis and interpretation of the results. In the current study, ARMA models are investigated for their potential to provide a comprehensive approach to data cleaning. Specific objectives are to: i) describe the cleaning algorithms and their integration with an existing open-source Matlab toolbox, ii) test the algorithms using a range of published data sets from two different instruments, and iii) recommend metrics to compare cleaning results. The recommended approach to detect and replace outliers in profiled velocity data is to use a spatial ‘seasonal’ filter that takes advantage of information available in neighboring cells and a low order ARMA model. Recommended data quality metrics are the spike frequency and the coefficients of the model. This approach is more precise than the most common current despiking method, offers a seamless method for generating replacement values that does not change the statistics of the velocity time series, and provides simple metrics with clear physical interpretation that can be used to compare the quality of different datasets.