The Power of Big Data and Data Analytics for AMI Data: A Case Study

In recent years, there has been a transformation in the value chain of different industrial sectors, like the electricity networks, with the appearance of smart grids. Currently, the underlying knowledge in raw data coming from numerous devices can mark a significant competitive advantage for utilities. It is the case of the Advanced Metering Infrastructure (AMI). Such technology gets user consumption characteristics at levels of detail that were previously not possible. In this context, the terms big data and data analytics become relevant, which are tools that allow using large volumes of information and the generation of valuable knowledge from raw data that can support data-driven decisions for operating on the grid. This paper presents the results of the big data implementation and data analytics techniques in a case study with smart metering data from the city of London. Implemented big data and data analytic techniques to show how to understand user consumption patterns on a broader horizon, the relationships with seasonal variables identify behaviors related to specific events and atypical consumptions. This knowledge helps support decision making about improving demand response programs and, in general, the planning and operation of the Smart Grid.

[1]  Ludmila Cherkasova,et al.  Automating Energy Demand Modeling and Forecasting Using Smart Meter Data , 2019, 2019 IEEE International Congress on Internet of Things (ICIOT).

[2]  Xinghuo Yu,et al.  Smart Electricity Meter Data Intelligence for Future Energy Systems: A Survey , 2016, IEEE Transactions on Industrial Informatics.

[3]  S. Fawcett,et al.  Data Science, Predictive Analytics, and Big Data: A Revolution that Will Transform Supply Chain Design and Management , 2013 .

[4]  Yogesh L. Simmhan,et al.  Cloud-Based Software Platform for Big Data Analytics in Smart Grids , 2013, Computing in Science & Engineering.

[5]  Walid G. Morsi,et al.  Nonintrusive Load Monitoring Using Wavelet Design and Machine Learning , 2016, IEEE Transactions on Smart Grid.

[6]  Yacine Rezgui,et al.  Random forests and artificial neural network for predicting daylight illuminance and energy consumption , 2017 .

[7]  Yu Yan,et al.  A fog computing solution for advanced metering infrastructure , 2016, 2016 IEEE/PES Transmission and Distribution Conference and Exposition (T&D).

[8]  Geza Joos,et al.  An effective feature extraction method in pattern recognition based high impedance fault detection , 2017, 2017 19th International Conference on Intelligent System Application to Power Systems (ISAP).

[9]  Yacine Rezgui,et al.  Trees vs Neurons: Comparison between random forest and ANN for high-resolution prediction of building energy consumption , 2017 .

[10]  Xiufeng Liu,et al.  A Hybrid ICT-Solution for Smart Meter Data Analytics , 2016, ArXiv.

[11]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[12]  Milan Prodanovic,et al.  State Forecasting and Operational Planning for Distribution Network Energy Management Systems , 2016, IEEE Transactions on Smart Grid.

[13]  Sousso Kelouwani,et al.  Approach in Nonintrusive Type I Load Monitoring Using Subtractive Clustering , 2017, IEEE Transactions on Smart Grid.

[14]  L. Suganthi,et al.  Energy models for demand forecasting—A review , 2012 .

[15]  Matteo Muratori,et al.  Big Data issues and opportunities for electric utilities , 2015 .

[16]  Ian Walker,et al.  How smart do smart meters need to be? , 2017 .

[17]  D. Nguyen,et al.  Big Data Analytics on a Smart Grid , 2016 .

[18]  Hamid Lesani,et al.  AMI data analytics; an investigation of the self-organizing maps capabilities in customers characterization and big data management , 2017, 2017 Smart Grid Conference (SGC).

[19]  Shiyin Zhong,et al.  A Frequency Domain Approach to Characterize and Analyze Load Profiles , 2012, IEEE Transactions on Power Systems.

[20]  Nadeem Javaid,et al.  An Accurate and Fast Converging Short-Term Load Forecasting Model for Industrial Applications in a Smart Grid , 2017, IEEE Transactions on Industrial Informatics.

[21]  Nikolay Kakanakov,et al.  Big data analytics in electricity distribution systems , 2017, 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[22]  Canbing Li,et al.  A Distributed Short-Term Load Forecasting Method Based on Local Weather Information , 2018, IEEE Systems Journal.

[23]  Bela Genge,et al.  A Brief Survey on Smart Grid Data Analysis in the Cloud , 2015 .

[24]  Jun Hu,et al.  Short-Term Load Forecasting With Deep Residual Networks , 2018, IEEE Transactions on Smart Grid.

[25]  Imdadullah Khan,et al.  Short Term Load Forecasting using Smart Meter Data , 2019, e-Energy.

[26]  James Christopher Foreman,et al.  Aggregation architecture for data reduction and privacy in advanced metering infrastructure , 2017, 2017 IEEE PES Innovative Smart Grid Technologies Conference - Latin America (ISGT Latin America).

[27]  Patrick J. Balducci,et al.  Using Smart Grids to Enhance Use of Energy-Efficiency and Renewable-Energy Technologies , 2011 .

[28]  K. P. Soman,et al.  Apache Spark a Big Data Analytics Platform for Smart Grid , 2015 .

[29]  Magnus Almgren,et al.  2016 Ieee International Conference on Big Data (big Data) Detecting Non-technical Energy Losses through Structural Periodic Patterns in Ami Data , 2022 .

[30]  H. Pao.  Comparing linear and nonlinear forecasts for Taiwan's electricity consumption , 2006 .

[31]  Subir Sen,et al.  Advanced metering infrastructure analytics — A Case Study , 2014, 2014 Eighteenth National Power Systems Conference (NPSC).

[32]  Zita Vale,et al.  A data-mining-based methodology to support MV electricity customers’ characterization , 2015 .

[33]  Aleksandr Ometov,et al.  Visualizing Big Data , 2016, Big Data Technologies and Applications.

[34]  Angshul Majumdar,et al.  Deep Sparse Coding for Non–Intrusive Load Monitoring , 2018, IEEE Transactions on Smart Grid.

[35]  Antonello Monti,et al.  A cloud-based smart metering infrastructure for distribution grid services and automation , 2017, Sustainable Energy, Grids and Networks.

[36]  Taxonomies Subgroup..  NIST Big Data Interoperability Framework:: volume 6, reference architecture version 3 , 2019 .

[37]  W. Fichtner,et al.  Electricity load profiles in Europe: The importance of household segmentation , 2014 .

[38]  Song Li,et al.  A Novel Wavelet-Based Ensemble Method for Short-Term Load Forecasting with Hybrid Neural Networks and Feature Selection , 2016, IEEE Transactions on Power Systems.

[39]  Xinghuo Yu,et al.  Advanced analytics for harnessing the power of smart meter big data , 2013, 2013 IEEE International Workshop on Inteligent Energy Systems (IWIES).

[40]  Javier Rosero Garcia,et al.  Advanced metering infrastructure in Colombia: benefits, challenges and opportunities , 2018 .

[41]  Akin Tascikaraoglu.  On Data-Driven Approaches for Demand Response , 2018 .

[42]  Nicolaus Henke,et al.  The age of analytics: competing in a data-driven world , 2016 .

[43]  Lukasz Golab,et al.  Smart Meter Data Analytics , 2017, ACM Trans. Database Syst..

[44]  Ram Rajagopal,et al.  Demand response targeting using big data analytics , 2013, 2013 IEEE International Conference on Big Data.

[45]  Taxonomies Subgroup..  NIST Big Data Interoperability Framework:: volume 3, use cases and general requirements version 3 , 2019 .

[46]  Assia Maamar,et al.  Machine learning Techniques for Energy Theft Detection in AMI , 2018 .

[47]  Houda Daki,et al.  Big Data management in smart grid: concepts, requirements and implementation , 2017, Journal of Big Data.

[48]  Wen-Shyong Yu,et al.  Data analysis of the smart meters and its applications in Tatung University , 2016, 2016 International Conference on Fuzzy Theory and Its Applications (iFuzzy).

[49]  Yogesh L. Simmhan,et al.  Improving Energy Use Forecast for Campus Micro-grids Using Indirect Indicators , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[50]  Yi Wang,et al.  Review of Smart Meter Data Analytics: Applications, Methodologies, and Challenges , 2018, IEEE Transactions on Smart Grid.

[51]  Robert W. Uluski,et al.  Realizing the smart grid of the future through AMI technology , 2009 .

[52]  Shanlin Yang,et al.  Big data driven smart energy management: From big data to big insights , 2016 .

[53]  Rob J. Hyndman,et al.  Hierarchical Probabilistic Forecasting of Electricity Demand With Smart Meter Data , 2020 .

[54]  Fei Jiang,et al.  Big data issues in smart grid – A review , 2017 .