Data Quality and Failures Characterization of Sensing Data in Environmental Applications

Environmental monitoring, targeting at discovering and understanding the environmental laws and changes, is one of the most important sensor network application domains. Environmental monitoring is one of the most important sensor network application domains. The success of those applications is determined by the quality of the collected data. Thus, it is crucial to carefully analyze the collected sensing data, which not only helps us understand the features of monitored field, but also unveil any limitations and opportunities that should be considered in future sensor system design. In this paper, we take an initial step and analyze one-month sensing data collected from a real-world water system surveillance application, focusing on the data similarity, data abnormality and failure patterns. Our major findings include: (1) Information similarity, including pattern similarity and numerical similarity, is very common, which provides a good opportunity to trade off energy efficiency and data quality; (2) Spatial and multi-modality correlation analysis provide a way to evaluate data integrity and to detect conflicting data that usually indicates appearances of sensor malfunction or interesting events; and (3) External harsh environmental conditions may be the most important factor on inflicting failures in environmental applications. Communication failures, mainly caused by lacking of synchronization, contribute the largest portion among all failure types.