Online and scalable data validation in advanced metering infrastructures

The shift from traditional to cyber-physical grids involves the deployment of Advanced Metering Infrastructures, networks of communication-enabled devices remotely controlled by utilities. Live information collected by these devices enables for applications such as demand/response, real-time pricing or intrusion detection, among others. In these scenarios, data validation is necessary in order to preprocess the noisy and lossy data produced by the devices and make it available to utilities' or third parties' applications. Challenges proper of data validation in this domain include the possibility of expressing validation rules specific to an Advanced Metering Infrastructure installation and analysis techniques that cope with the large and fluctuating volume of data produced by the devices. In this paper, we discuss and provide evidence of the online, scalable and expressive validation analysis enabled by the data streaming processing paradigm. Based on a prototype implementation on top of the Storm processing engine and using data from a real-world Advanced Metering Infrastructure, we show that streaming-based validation rules enable for the analysis of thousands of meters per second and only incur in small latency penalties in the order of milliseconds.

[1]  Philippe Bonnet,et al.  Towards Sensor Database Systems , 2001, Mobile Data Management.

[2]  H. Allcott,et al.  Rethinking Real Time Electricity Pricing , 2011 .

[3]  Miao Pan,et al.  Decentralized Coordination of Energy Utilization for Residential Households in the Smart Grid , 2013, IEEE Transactions on Smart Grid.

[4]  Valentin Tudor,et al.  Analysis of the impact of data granularity on privacy for the smart grid , 2013, WPES.

[5]  Yehuda Naveh,et al.  Analysis of advanced meter infrastructure data of water consumption in apartment buildings , 2013, KDD.

[6]  Yogesh L. Simmhan,et al.  Adaptive rate stream processing for smart grid applications on clouds , 2011, ScienceCloud '11.

[7]  Claudio Soriente,et al.  StreamCloud: An Elastic and Scalable Data Streaming System , 2012, IEEE Transactions on Parallel and Distributed Systems.

[8]  William H. Sanders,et al.  AMI threats, intrusion detection requirements and deployment recommendations , 2012, 2012 IEEE Third International Conference on Smart Grid Communications (SmartGridComm).

[9]  João Gama,et al.  Stream-Based Electricity Load Forecast , 2007, PKDD.

[10]  S. Mauw,et al.  Specification-based intrusion detection for advanced metering infrastructures , 2022 .

[11]  Valentin Tudor,et al.  Remote Control of Smart Meters: Friend or Foe? , 2011, 2011 Seventh European Conference on Computer Network Defense.

[12]  Haifeng Jiang,et al.  Photon: fault-tolerant and scalable joining of continuous data streams , 2013, SIGMOD '13.

[13]  Odej Kao,et al.  Processing smart meter data streams in the cloud , 2011, 2011 2nd IEEE PES International Conference and Exhibition on Innovative Smart Grid Technologies.

[14]  Michael Stonebraker,et al.  Monitoring Streams - A New Class of Data Management Applications , 2002, VLDB.

[15]  Marina Papatriantafilou,et al.  STONE: a stream-based DDoS defense framework , 2013, SAC '13.

[16]  Ying Xing,et al.  The Design of the Borealis Stream Processing Engine , 2005, CIDR.

[17]  John R. Williams,et al.  Securing Advanced Metering Infrastructure Using Intrusion Detection System with Data Stream Mining , 2012, PAISI.

[18]  Frederick Reiss,et al.  TelegraphCQ: continuous dataflow processing , 2003, SIGMOD '03.

[19]  Qiang Chen,et al.  Aurora : a new model and architecture for data stream management ) , 2006 .

[20]  Michael Stonebraker,et al.  The 8 requirements of real-time stream processing , 2005, SGMD.

[21]  Matti Lehtonen,et al.  Near real time energy monitoring for end users: Requirements and sample applications , 2011, 2011 IEEE International Conference on Smart Grid Communications (SmartGridComm).