Hadoop-based framework for big data analysis of synchronised harmonics in active distribution network

Synchronised harmonic analysis that utilises GPS signals to synchronise harmonic measurements enables advanced power quality analysis in smart grid. However, it is difficult to put this promising technology into practice due to an extremely high requirement on data storage and processing. To overcome this problem, this study presents a novel Hadoop-based framework for big data analysis of synchronised harmonics in distribution networks. The proposed framework facilitates harmonic data processing associated with data storage and advanced analysis based on big data techniques. Under this framework, a big matrix multiplication algorithm based on MapReduce programming model solving the harmonic state estimation problem is deployed in the stage of data processing. A MapReduce-based harmonics distortion calculation is implemented as an advanced analysis for harmonics. Comprehensive numerical studies are carried out to analyse the characteristics of the proposed algorithm and verify the effectiveness of the proposed framework.

[1]  Ozgul Salor,et al.  Nationwide real-time monitoring system for electrical quantities and power quality of the electricity transmission system , 2011 .

[2]  Yonggang Wen,et al.  Toward Scalable Systems for Big Data Analytics: A Technology Tutorial , 2014, IEEE Access.

[3]  A.P.S. Meliopoulos,et al.  Harmonic monitoring system via GPS-synchronized measurements - update and new developments , 2006, 2006 IEEE Power Engineering Society General Meeting.

[4]  A Q Huang,et al.  The Future Renewable Electric Energy Delivery and Management (FREEDM) System: The Energy Internet , 2011, Proceedings of the IEEE.

[5]  Ameet Talwalkar,et al.  MLlib: Machine Learning in Apache Spark , 2015, J. Mach. Learn. Res..

[6]  Salvatore Nuccio,et al.  A self-synchronizing instrument for harmonic source detection in power systems , 2005, IEEE Transactions on Instrumentation and Measurement.

[7]  Nelson Kagan,et al.  Harmonic state estimation through optimal monitoring systems , 2013, IEEE Transactions on Smart Grid.

[8]  Yogesh L. Simmhan,et al.  Cloud-Based Software Platform for Big Data Analytics in Smart Grids , 2013, Computing in Science & Engineering.

[9]  Neville R. Watson,et al.  Continuous harmonic state estimation of power systems , 1996 .

[10]  Fabrizio Pilo,et al.  Optimal Allocation of Multichannel Measurement Devices for Distribution State Estimation , 2009, IEEE Transactions on Instrumentation and Measurement.

[11]  Yonghua Song,et al.  Stochastic Bi-level Trading Model for an Active Distribution Company with DGs and Interruptible Loads , 2017 .

[12]  Junyong Liu,et al.  Parallel Detrended Fluctuation Analysis for Fast Event Detection on Massive PMU Data , 2015, IEEE Transactions on Smart Grid.

[13]  Nadarajah Mithulananthan,et al.  Harmonic impact of high penetration photovoltaic system on unbalanced distribution networks - learning from an urban photovoltaic network , 2016 .

[14]  Aurelio Medina,et al.  Time-domain harmonic state estimation based on the kalman filter poincare map and extrapolation to the limit cycle , 2012 .

[15]  Kit Po Wong,et al.  Probabilistic Forecasting of Wind Power Generation Using Extreme Learning Machine , 2014, IEEE Transactions on Power Systems.

[16]  Wu He,et al.  Internet of Things in Industries: A Survey , 2014, IEEE Transactions on Industrial Informatics.

[17]  Ram Rajagopal,et al.  Smart Meter Driven Segmentation: What Your Consumption Says About You , 2013, IEEE Transactions on Power Systems.

[18]  Luigi Vanfretti,et al.  The OpenPMU Platform for Open-Source Phasor Measurements , 2013, IEEE Transactions on Instrumentation and Measurement.

[19]  Kit Po Wong,et al.  Optimal Prediction Intervals of Wind Power Generation , 2014, IEEE Transactions on Power Systems.

[20]  Jin Lin,et al.  Optimal Meter Placement for Distribution Network State Estimation: A Circuit Representation Based MILP Approach , 2016, IEEE Transactions on Power Systems.

[21]  Michael D. Ernst,et al.  HaLoop , 2010, Proc. VLDB Endow..

[22]  Kenneth A. Loparo,et al.  Big data analytics in power distribution systems , 2015, 2015 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT).

[23]  Dong Liu,et al.  Parallel Matrix Multiplication Algorithm Based on Vector Linear Combination Using MapReduce , 2013, 2013 IEEE Ninth World Congress on Services.

[24]  Mario Paolone,et al.  Synchronized Phasors Monitoring During the Islanding Maneuver of an Active Distribution Network , 2011, IEEE Transactions on Smart Grid.

[25]  Hoang Le-Huy,et al.  Modeling and simulation of the propagation of harmonics in electric power networks. I: Concepts, models, and simulation techniques. Discussion , 1996 .

[26]  M. Wache,et al.  Application of Synchrophasor Measurements for distribution networks , 2011, 2011 IEEE Power and Energy Society General Meeting.

[27]  Fan Zhang,et al.  Power system harmonic state estimation , 1994 .

[28]  Huaiwei Liao,et al.  Power System Harmonic State Estimation and Observability Analysis via Sparsity Maximization , 2007, IEEE Transactions on Power Systems.

[29]  Yonghua Song,et al.  Model Predictive Control of LPC-Looped Active Distribution Network With High Penetration of Distributed Generation , 2017, IEEE Transactions on Sustainable Energy.

[30]  Xindong Wu,et al.  Database classification for multi-database mining , 2005, Inf. Syst..

[31]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[32]  Task Force,et al.  Characteristics and Modeling of Harmonic Sources-Power Electronic Devices , 2001, IEEE Power Engineering Review.

[33]  Graham J. Williams,et al.  Big Data Opportunities and Challenges: Discussions from Data Analytics Perspectives [Discussion Forum] , 2014, IEEE Computational Intelligence Magazine.

[34]  P.J.M. Heskes,et al.  Harmonic interaction between a large number of distributed power inverters and the distribution network , 2004, IEEE Transactions on Power Electronics.

[35]  Mukhtiar Singh,et al.  Grid Interconnection of Renewable Energy Sources at the Distribution Level With Power-Quality Improvement Features , 2011, IEEE Transactions on Power Delivery.

[36]  Jianhua Zhang,et al.  Distributed Optimization Algorithms for Wide-Area Oscillation Monitoring in Power Systems Using Interregional PMU-PDC Architectures , 2015, IEEE Transactions on Smart Grid.

[37]  J. Matsuki,et al.  Power system harmonic analysis using state-estimation method for Japanese field data , 2005, IEEE Transactions on Power Delivery.

[38]  Deepak Divan,et al.  Smart “Stick-on” Sensors for the Smart Grid , 2012, IEEE Transactions on Smart Grid.

[39]  Carlo Muscas,et al.  A PMU for the Measurement of Synchronized Harmonic Phasors in Three-Phase Distribution Networks , 2009, IEEE Transactions on Instrumentation and Measurement.

[40]  S.-J.S. Tsai,et al.  Synchronized Power-Quality Measurement Network With LAMP , 2009, IEEE Transactions on Power Delivery.

[41]  Hong Liu,et al.  A diagnostic method for distribution networks based on power supply safety standards , 2016 .

[42]  J. Arrillaga,et al.  An adaptive Kalman filter for dynamic harmonic state estimation and harmonic injection tracking , 2005, IEEE Transactions on Power Delivery.

[43]  W. H. Kersting.  Radial distribution test feeders , 1991 .

[44]  Yonghua Song,et al.  Optimal Cloud Computing Resource Allocation for Demand Side Management in Smart Grid , 2017, IEEE Transactions on Smart Grid.

[45]  Santiago Grijalva,et al.  Leveraging AMI Data for Distribution System Model Calibration and Situational Awareness , 2015, IEEE Transactions on Smart Grid.