Anomaly Detection System for Water Networks in Northern Ethiopia Using Bayesian Inference

For billions of people living in remote and rural communities in the developing countries, small water systems are the only source of clean drinking water. Due to the rural nature of such water systems, site visits may occur infrequently. This means broken water systems can remain in a malfunctioning state for months, forcing communities to return to drinking unsafe water. In this work, we present a novel two-level anomaly detection system aimed to detect malfunctioning remote sensored water hand-pumps, allowing for a proactive approach to pump maintenance. To detect anomalies, we need a model of normal water usage behavior first. We train a multilevel probabilistic model of normal usage using approximate variational Bayesian inference to obtain a conditional probability distribution over the hourly water usage data. We then use this conditional distribution to construct a level-1 scoring function for each hourly water observation and a level-2 scoring function for each pump. Probabilistic models and Bayesian inference collectively were chosen for their ability to capture the high temporal variability in the water usage data at the individual pump level as well as their ability to estimate interpretable model parameters. Experimental results in this work have demonstrated that the pump scoring function is able to detect malfunctioning sensors as well as a change in water usage behavior allowing for a more responsive and proactive pump system maintenance.

[1]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[2]  Michael Goldstein,et al.  A Bayesian analysis of fluid flow in pipe‐lines , 2001 .

[3]  Rehan Sadiq,et al.  Integrated Decision Support System for Prognostic and Diagnostic Analyses of Water Distribution System Failures , 2016, Water Resources Management.

[4]  Pedro Lee,et al.  The influence of non-uniform blockages on transient wave behavior and blockage detection in pressurized water pipelines , 2017 .

[5]  C. F. Jeff Wu,et al.  On Prediction Properties of Kriging: Uniform Error Bounds and Robustness , 2017, Journal of the American Statistical Association.

[6]  David A. Clifton,et al.  Distributed Inference Condition Monitoring System for Rural Infrastructure in the Developing World , 2019, IEEE Sensors Journal.

[7]  Yeou-Koung Tung,et al.  Probabilistic Analysis of Transient Design for Water Supply Systems , 2010 .

[8]  T. Foster,et al.  Functionality of handpump water supplies: a review of data from sub-Saharan Africa and the Asia-Pacific region , 2020, International Journal of Water Resources Development.

[9]  Evan A Thomas,et al.  Ensemble machine learning and forecasting can achieve 99% uptime for rural handpumps , 2017, PloS one.

[10]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[11]  Dustin Tran,et al.  Automatic Differentiation Variational Inference , 2016, J. Mach. Learn. Res..

[12]  Pedro J. Lee,et al.  Transient-Based Frequency Domain Method for Dead-End Side Branch Detection in Reservoir Pipeline-Valve Systems , 2016 .

[13]  Antonio Candelieri Clustering and Support Vector Regression for Water Demand Forecasting and Anomaly Detection , 2017 .

[14]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[15]  T. Foster Predictors of sustainability for community-managed handpumps in sub-Saharan Africa: evidence from Liberia, Sierra Leone, and Uganda. , 2013, Environmental science & technology.

[16]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[17]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[18]  Hongwei Zhang,et al.  An assessment model of water pipe condition using Bayesian inference , 2010 .

[19]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[20]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[21]  Michael B Fisher,et al.  Understanding handpump sustainability: Determinants of rural water source functionality in the Greater Afram Plains region of Ghana† , 2015, Water resources research.

[22]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[23]  Philippe Hartemann,et al.  Estimating the impact on health of poor reliability of drinking water interventions in developing countries. , 2009, The Science of the total environment.

[24]  Richard Mounce,et al.  Novelty detection for time series data analysis in water distribution systems using support vector machines , 2011 .