A Weekly Load Data Mining Approach Based on Hidden Markov Model

With the development of advanced metering infrastructure, massive smart meter readings are generated and stored in smart grids, which makes it possible for detecting of tremendous social value embedded in load data. The majority of the existing load data mining works are performed on the daily time scale without adequate consideration of load information between the days. To better describe the power consumption characteristics of users, a data mining approach based on the weekly load curves is proposed in this paper. First, the piecewise aggregate approximation technique is utilized to reduce the dimensions of the raw weekly load data. Then, a Davies–Bouldin index-based adaptive k-means algorithm is proposed to cluster the studied users into several groups. Finally, a hidden Markov model describing the probabilistic transitions of different load levels is established for each cluster to extract the representative dynamic weekly load features. A feasible tool based on dynamic characteristics of load patterns is invented to evaluate the short-term load forecasting methods, which realizes the pre-check for the forecasting results without future real measurements in the forecasting horizon. Case studies on a real dataset demonstrate that the proposed method is capable of extracting weekly load characteristics of users.

[1]  Lei Chen,et al.  Research on load classification based on user's typical daily load curve , 2017, 2017 IEEE Conference on Energy Internet and Energy System Integration (EI2).

[2]  Ahmet Teke,et al.  A state-of-the-art review of artificial intelligence techniques for short-term electric load forecasting , 2017, 2017 6th International Youth Conference on Energy (IYCE).

[3]  Prasanta K. Jana,et al.  An effective Multi-Objective task scheduling algorithm using Min-Max normalization in cloud computing , 2016, 2016 2nd International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT).

[4]  Qingqing Huang,et al.  Dynamic Detection of Transmission Line Outages Using Hidden Markov Models , 2016, IEEE Transactions on Power Systems.

[5]  Furong Li,et al.  Development of Low Voltage Network Templates—Part II: Peak Load Estimation by Clusterwise Regression , 2015, IEEE Transactions on Power Systems.

[6]  Furong Li,et al.  Development of Low Voltage Network Templates—Part I: Substation Clustering and Classification , 2015, IEEE Transactions on Power Systems.

[7]  Chongqing Kang,et al.  Load profiling and its application to demand response: A review , 2015 .

[8]  Zong-chang Yang,et al.  Electric Load Movement Evaluation and Forecasting Based on the Fourier-Series Model Extend in the Least-Squares Sense , 2015 .

[9]  S. Varuna,et al.  An integration of k-means clustering and naïve bayes classifier for Intrusion Detection , 2015, 2015 3rd International Conference on Signal Processing, Communication and Networking (ICSCN).

[10]  Papia Ray,et al.  Hybrid methodology for short-term load forecasting , 2014, 2014 IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES).

[11]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[12]  Li Zhang,et al.  The optimized K-means algorithms for improving randomly-initialed midpoints , 2013, Proceedings of 2013 2nd International Conference on Measurement, Information and Control.

[13]  Nazih M. Abu-Shikhah,et al.  Medium-term electric load forecasting using singular value decomposition , 2011 .

[14]  G. Chicco,et al.  Renyi entropy-based classification of daily electrical load patterns , 2010 .

[15]  James Bailey,et al.  Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance , 2010, J. Mach. Learn. Res..

[16]  Ming-Hu Ha,et al.  Support vector machine based on half-suppressed fuzzy c-means clustering , 2009, 2009 International Conference on Machine Learning and Cybernetics.

[17]  Ying Tan,et al.  A Framework of Stochastic Power Management Using Hidden Markov Model , 2008, 2008 Design, Automation and Test in Europe.

[18]  N.D. Hatziargyriou,et al.  Two-Stage Pattern Recognition of Load Curves for Classification of Electricity Customers , 2007, IEEE Transactions on Power Systems.

[19]  Z. Vale,et al.  Data Mining Contributions to Characterize MV Consumers and to Improve the Suppliers-Consumers Settlements , 2007, 2007 IEEE Power Engineering Society General Meeting.

[20]  Jieping Ye,et al.  Least squares linear discriminant analysis , 2007, ICML '07.

[21]  G. Chicco,et al.  Comparisons among clustering techniques for electricity customer classification , 2006, IEEE Transactions on Power Systems.

[22]  Md. Rafiul Hassan,et al.  Stock market forecasting using hidden Markov model: a new approach , 2005, 5th International Conference on Intelligent Systems Design and Applications (ISDA'05).

[23]  P. Postolache,et al.  Load pattern-based classification of electricity customers , 2004, IEEE Transactions on Power Systems.

[24]  Eamonn J. Keogh,et al.  A symbolic representation of time series, with implications for streaming algorithms , 2003, DMKD '03.

[25]  Dagmar Niebur,et al.  Load profile estimation in electric transmission networks using independent component analysis , 2003 .

[26]  K. Becker,et al.  Analysis of microarray data using Z score transformation. , 2003, The Journal of molecular diagnostics : JMD.

[27]  H. Kobayashi,et al.  An efficient forward-backward algorithm for an explicit-duration hidden Markov model , 2003, IEEE Signal Processing Letters.

[28]  Kee-Choon Kwon,et al.  Hidden Markov model‐based real‐time transient identifications in nuclear power plants , 2002, Int. J. Intell. Syst..

[29]  Eamonn J. Keogh,et al.  Dimensionality Reduction for Fast Similarity Search in Large Time Series Databases , 2001, Knowledge and Information Systems.

[30]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[31]  Magdy M. A. Salama,et al.  Review of electric load forecasting methods , 1997, CCECE '97. Canadian Conference on Electrical and Computer Engineering. Engineering Innovation: Voyage of Discovery. Conference Proceedings.

[32]  Yoshifusa Ito,et al.  Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory , 1991, Neural Networks.

[33]  R. Hart,et al.  Demonstration of a Jacobi Polynomial Representation of a Kronecker Delta Function. , 1985 .

[34]  A. M. Zakharin.  Some Markov models of systems with partial aftereffect , 1980 .

[35]  D. F. Andrews,et al.  A Robust Method for Multiple Linear Regression , 1974 .

[36]  Jin Cui,et al.  A pattern recognition method for electric nose based on kernel-principal component analysis(PCA) and support vector machine(SVM) , 2012 .

[37]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[38]  Enrico Carpaneto,et al.  Electricity customer classification using frequency–domain load pattern data , 2006 .

[39]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.