Error Detection in Ocean Data Considering Spatial Autocorrelation

Error detection in ocean data is difficult because characteristics of the ocean data are different among ocean areas. For now, the accurate error detection depends on visual checks by ocean data technicians. However, human resources are limited and their skills are not uniform, which makes it difficult to deliver accurate and uniformly quality-controlled ocean data. In this work, we propose a framework for an automated error detection in the ocean data, that is applicable for unknown types of errors, considering spatial autocorrelation. Our proposal framework consists of a training data selecting phase to take the spatial autocorrelation into consideration and an error detection phase. As a result of empirical experiments, we found the effective combinations of features, training data selecting methods and anomaly detection methods, regarding the ocean characteristics. In addition, our proposal training data selecting method worked efficiently, even when the number of training data was few around test data.

[1]  Ken-ichi Fukui,et al.  Error detection of oceanic observation data using sequential labeling , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[2]  Fei Tony Liu,et al.  Isolation-Based Anomaly Detection , 2012, TKDD.

[3]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[4]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.