A methodology and a software tool for sensor data validation/reconstruction : application to the Catalonia regional water network

In this paper, a sensor data validation/reconstruction methodology applicable to water networks and its implementation by means of a software tool are presented. The aim is to guarantee that the sensor data are reliable and complete in case that sensor faults occur. The availability of such dataset is of paramount importance in order to successfully use the sensor data for further tasks e.g. water billing, network efficiency assessment, leak localization and real-time operational control. The methodology presented here is based on a sequence of tests and on the combined use of spatial models (SM) and time series models (TSM) applied to the sensors used for real-time monitoring and control of the water network. Spatial models take advantage of the physical relations between different system variables (e.g. flow and level sensors in hydraulic systems) while time series models take advantage of the temporal redundancy of the measured variables (here by means of a Holt–Winters (HW) time series model). First, the data validation approach, based on several tests of different complexity, is described to detect potential invalid or missing data. Then, the reconstruction process is based on a set of spatial and time series models used to reconstruct the missing/invalid data with the model estimation providing the best fit. A software tool implementing the proposed data validation and reconstruction methodology is also described. Finally, results obtained applying the proposed methodology to a real case study based on the Catalonia regional water network is used to illustrate its performance.

[1]  P. Vanrolleghem,et al.  Real time control of urban wastewater systems: where do we stand today? , 2004 .

[2]  R. Brown.  Statistical forecasting for inventory control , 1960 .

[3]  Steven X. Ding,et al.  Model-based Fault Diagnosis Techniques: Design Schemes, Algorithms, and Tools , 2008 .

[4]  Steven C. Wheelwright,et al.  Forecasting methods and applications. , 1979 .

[5]  F. Edthofer,et al.  Reliable online water quality monitoring as basis for fault tolerant control , 2010, 2010 Conference on Control and Fault-Tolerant Systems (SysTol).

[6]  Ramon Pérez Magrané,et al.  Leakage location in water distribution networks based on correlation measurement of pressure sensors , 2011 .

[7]  Peter Steen Mikkelsen,et al.  Quality control of rain data used for urban runoff systems , 1997 .

[8]  E. S. Gardner.  EXPONENTIAL SMOOTHING: THE STATE OF THE ART, PART II , 2006 .

[9]  Paul G. Plöger,et al.  Model-Based Fault Diagnosis Techniques for Mobile Robots , 2016 .

[10]  Christophe Croux,et al.  Robust Forecasting with Exponential and Holt-Winters Smoothing , 2007 .

[11]  Vasilis Kanakoudis,et al.  Pipe Networks Risk Assessment Based on Survival Analysis , 2011 .

[12]  Vicenç Puig,et al.  Methodology for leakage isolation using pressure sensitivity analysis in water distribution networks , 2011 .

[13]  James W. Taylor,et al.  Triple seasonal methods for short-term electricity demand forecasting , 2010, Eur. J. Oper. Res..

[14]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part I: Quantitative model-based methods , 2003, Comput. Chem. Eng..

[15]  Francesco Piazza,et al.  A review of datasets and load forecasting techniques for smart natural gas and water grids: Analysis and experiments , 2015, Neurocomputing.

[16]  Jose A. Romagnoli,et al.  Data Processing and Reconciliation for Chemical Process Operations , 1999 .

[17]  Vicenç Puig,et al.  Validation and reconstruction of flow meter data in the Barcelona water distribution network , 2010 .

[18]  Magdalene Marinaki,et al.  Optimal Real-time Control of Sewer Networks , 2005 .

[19]  Vicenç Puig,et al.  Combining contaminant event diagnosis with data validation/reconstruction: Application to smart buildings , 2014, 22nd Mediterranean Conference on Control and Automation.

[20]  K M Tsang.  Sensor data validation using gray models. , 2003, ISA transactions.

[21]  Peter R. Winters,et al.  Forecasting Sales by Exponentially Weighted Moving Averages , 1960 .

[22]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[23]  Zoran Kapelan,et al.  Improved real-time data anomaly detection using context classification , 2011 .

[24]  M Mourad,et al.  A method for automatic validation of long time series of data in urban hydrology. , 2002, Water science and technology : a journal of the International Association on Water Pollution Research.

[25]  D Burnell.  Auto-validation of district meter data , 2003 .

[26]  V. K. Kanakoudis,et al.  The role of leaks and breaks in water networks: technical and economical solutions , 2001 .

[27]  Fatiha Nejjari,et al.  Leak Localization in Water Networks: A Model-Based Methodology Using Pressure Sensors Applied to a Real Network in Barcelona [Applications of Control] , 2014, IEEE Control Systems.

[28]  Santiago Espin,et al.  Methodology of a data validation and reconstructions tool to improve the realiability of the water network supervision , 2010 .

[29]  Peter A. Vanrolleghem,et al.  Real-time control of urban wastewater systems - Where do we stand today ? , 2002 .

[30]  Vasilis Kanakoudis,et al.  Water pipe network reliability assessment using the DAC method , 2011 .

[31]  Vicenç Puig,et al.  Inconsistent sensor data detection/correction: Application to environmental systems , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[32]  Shankar Narasimhan,et al.  Data reconciliation & gross error detection: an intelligent use of process data , 1999 .

[33]  J. W. Taylor,et al.  Short-term electricity demand forecasting using double seasonal exponential smoothing , 2003, J. Oper. Res. Soc..

[34]  Garth P. McCormick,et al.  Communications to the Editor--Exponential Forecasting: Some New Variations , 1969 .

[35]  Vicenç Puig,et al.  Fault diagnosis and fault tolerant control using set-membership approaches: Application to real case studies , 2010, Int. J. Appl. Math. Comput. Sci..

[36]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.