Multi-objective ensemble forecasting with an application to power transformers

Abstract In this paper we present an ensemble time series forecasting algorithm using evolutionary multi-objective optimization algorithms to predict dissolved gas contents in power transformers. In this method, the correlation between each individual dissolved gas and other transformers’ features such as temperature characteristics and loading history is first determined. Then, a non-linear principal component analysis (NLPCA) technique is applied to extract the most effective time series from the highly correlated features. Afterwards, the forecasting algorithms are trained using a cross validation technique. In addition, evolutionary multi-objective optimization algorithms are used to select the most accurate and diverse group of forecasting algorithms to construct an ensemble. Finally, the selected ensemble is examined to predict the value of the dissolved gases on the testing set. The results of one day, two day, three day, and four day ahead forecasting are presented which show higher accuracy and reliability of the proposed method compared with other statistical methods.

[1]  Mehdi Khashei,et al.  A novel hybridization of artificial neural networks and ARIMA models for time series forecasting , 2011, Appl. Soft Comput..

[2]  Peter Tiño,et al.  Managing Diversity in Regression Ensembles , 2005, J. Mach. Learn. Res..

[3]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[4]  Yu Cheng,et al.  Seasonal and trend time series forecasting based on a quasi-linear autoregressive model , 2014, Appl. Soft Comput..

[5]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[6]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[7]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[8]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[9]  Mingjun Wang,et al.  Particle swarm optimization-based support vector machine for forecasting dissolved gases content in power transformer oil , 2009 .

[10]  Tapan Kumar Saha,et al.  Review of modern diagnostic techniques for assessing insulation condition in aged transformers , 2003 .

[11]  Meng-Hui Wang.  Grey-Extension Method for Incipient Fault Forecasting of Oil-Immersed Power Transformer , 2004 .

[12]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[13]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[14]  Hussein A. Abbass,et al.  Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[15]  Kin Keung Lai,et al.  A neural-network-based nonlinear metamodeling approach to financial time series forecasting , 2009, Appl. Soft Comput..

[16]  Abdolrahman Peimankar,et al.  Evolutionary multi-objective fault diagnosis of power transformers , 2017, Swarm Evol. Comput..

[17]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[18]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[19]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[20]  Mikel Galar,et al.  Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy , 2016, Appl. Soft Comput..

[21]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[22]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[23]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[24]  Václav Snásel,et al.  Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming , 2017, Appl. Soft Comput..

[25]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[26]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[27]  Richard A. Watson,et al.  Reducing Local Optima in Single-Objective Problems by Multi-objectivization , 2001, EMO.

[28]  Ponnuthurai N. Suganthan,et al.  Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [Review Article] , 2016, IEEE Computational Intelligence Magazine.

[29]  Kazutoshi Sakakibara,et al.  Multi-objective approaches in a single-objective optimization environment , 2005, 2005 IEEE Congress on Evolutionary Computation.

[30]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[31]  Xin Yao,et al.  Ensemble Learning Using Multi-Objective Evolutionary Algorithms , 2006, J. Math. Model. Algorithms.

[32]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[33]  Qingfu Zhang,et al.  Multiobjective evolutionary algorithms: A survey of the state of the art , 2011, Swarm Evol. Comput..

[34]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[35]  Magdy M. A. Salama,et al.  Asset management techniques for transformers , 2010 .

[36]  Ling Tang,et al.  LSSVR ensemble learning with uncertain parameters for crude oil price forecasting , 2017, Appl. Soft Comput..

[37]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[38]  Robert Sabourin,et al.  Single and Multi-Objective Genetic Algorithms for the Selection of Ensemble of Classifiers , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[39]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[40]  Nikolaos Kourentzes,et al.  Neural network ensemble operators for time series forecasting , 2014, Expert Syst. Appl..

[41]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[42]  Ponnuthurai Nagaratnam Suganthan,et al.  Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting , 2017, Appl. Soft Comput..

[43]  Kunikazu Kobayashi,et al.  Time series forecasting using a deep belief network with restricted Boltzmann machines , 2014, Neurocomputing.

[44]  Wenyu Zhang,et al.  Short-term wind speed forecasting based on a hybrid model , 2013, Appl. Soft Comput..

[45]  Joachim Selbig,et al.  Non-linear PCA: a missing data approach , 2005, Bioinform..

[46]  Yu Sun,et al.  Forecasting dissolved gases content in power transformer oil based on support vector machine with genetic algorithm , 2008 .

[47]  K. D. Srivastava,et al.  Review of condition assessment of power transformers in service , 2002 .

[48]  B. Bowerman,et al.  Forecasting, time series, and regression : an applied approach , 2005 .

[49]  Huanhuan Chen,et al.  Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[50]  Reza Ebrahimpour,et al.  Combining features of negative correlation learning with mixture of experts in proposed ensemble methods , 2012, Appl. Soft Comput..

[51]  Lijun Yang,et al.  Particle swarm optimization-least squares support vector regression based forecasting model on dissolved gases in oil-filled power transformers , 2011 .

[52]  Naonori Ueda,et al.  Generalization error of ensemble estimators , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[53]  A. H. El-Hag,et al.  Artificial neural networks with stepwise regression for predicting transformer oil furan content , 2012, IEEE Transactions on Dielectrics and Electrical Insulation.

[54]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[55]  C. Holt.  Author's retrospective on ‘Forecasting seasonals and trends by exponentially weighted moving averages’ , 2004 .

[56]  Abdolrahman Peimankar,et al.  Ensemble classifier selection using multi-objective PSO for fault diagnosis of power transformers , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[57]  E. Gockenbach,et al.  Asset-Management of Transformers Based on Condition Monitoring and Standard Diagnosis [Feature Article] , 2008, IEEE Electrical Insulation Magazine.

[58]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[59]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[60]  B. Eswara Reddy,et al.  A moving-average filter based hybrid ARIMA-ANN model for forecasting time series data , 2014, Appl. Soft Comput..

[61]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[62]  M M A Salama,et al.  A Techno-Economic Method for Replacing Transformers , 2011, IEEE Transactions on Power Delivery.

[63]  Qinghua Hu,et al.  Transfer learning for short-term wind speed prediction with deep neural networks , 2016 .

[64]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[65]  Le Zhang,et al.  Ensemble deep learning for regression and time series forecasting , 2014, 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL).

[66]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.