Approximate ideal multi-objective solution Q(λ) learning for optimal carbon-energy combined-flow in multi-energy power systems

Abstract This paper proposes a novel approximate ideal multi-objective solution Q(λ) learning for optimal carbon-energy combined-flow in multi-energy power systems. The carbon emissions, fuel cost, active power loss, voltage deviation and carbon emission loss are chosen as the optimization objectives, which are simultaneously optimized by five different Q-value matrices. The dynamic optimal weight of each objective is calculated online from the entire Q-value matrices such that the greedy action policy can be obtained. Case studies are carried out to evaluate the optimization performance for carbon-energy combined-flow in an IEEE 118-bus system and the regional power grid of southern China.

[1]  Yuehua Huang,et al.  A new quantum inspired chaotic artificial bee colony algorithm for optimal power flow problem , 2015 .

[2]  C. Kroeze.  N2O from animal waste. Methodology according to IPCC Guidelines for National Greenhouse Gas Inventories. , 1997 .

[3]  Jing Peng,et al.  Incremental multi-step Q-learning , 1994, Machine Learning.

[4]  Q. Henry Wu,et al.  Group Search Optimizer: An Optimization Algorithm Inspired by Animal Searching Behavior , 2009, IEEE Transactions on Evolutionary Computation.

[5]  Gang Zhang,et al.  Quantitative assessment on the cloning efficiencies of lentiviral transfer vectors with a unique clone site , 2012, Scientific Reports.

[6]  M. A. Abido,et al.  Optimal power flow using particle swarm optimization , 2002 .

[7]  Ahmed Bilal Awan,et al.  Combined emission economic dispatch of power system including solar photo voltaic generation , 2015 .

[8]  Hua Wei,et al.  An interior point nonlinear programming for optimal power flow problems with a novel data structure , 1997 .

[9]  Janusz Bialek,et al.  Tracing the flow of electricity , 1996 .

[10]  Tao Yu,et al.  Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step $Q(\lambda)$ Learning , 2011, IEEE Transactions on Power Systems.

[11]  Qixin Chen,et al.  Power Generation Expansion Planning Model Towards Low-Carbon Economy and Its Application in China , 2010, IEEE Transactions on Power Systems.

[12]  Tao Yu,et al.  Distributed multi-step Q(λ) learning for Optimal Power Flow of large-scale power grids , 2012 .

[13]  M. E. El-Hawary,et al.  Optimal Distributed Generation Allocation and Sizing in Distribution Systems via Artificial Bee Colony Algorithm , 2011, IEEE Transactions on Power Delivery.

[14]  Dewen Hu,et al.  Multiobjective Reinforcement Learning: A Comprehensive Overview , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Tao Yu,et al.  Equilibrium-Inspired Multiple Group Search Optimization With Synergistic Learning for Multiobjective Electric Power Dispatch , 2013, IEEE Transactions on Power Systems.

[16]  Chun-Yao Lee,et al.  Unit commitment with energy dispatch using a computationally efficient encoding structure , 2011 .

[17]  H.M. Khodr,et al.  Ant colony system algorithm for the planning of primary distribution circuits , 2004, IEEE Transactions on Power Systems.

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  Geoffrey P. Hammond,et al.  The prospects for coal-fired power plants with carbon capture and storage: A UK perspective , 2014 .

[20]  Chuangxin Guo,et al.  A multiagent-based particle swarm optimization approach for optimal reactive power dispatch , 2005 .

[21]  Xiaohui Yuan,et al.  Application of quantum-inspired binary gravitational search algorithm for thermal unit commitment with wind power integration , 2014 .

[22]  John N. Tsitsiklis,et al.  Asynchronous Stochastic Approximation and Q-Learning , 1994, Machine Learning.

[23]  Tao Yu,et al.  Stochastic optimal generation command dispatch based on improved hierarchical reinforcement learning approach , 2011 .

[24]  D. Ernst,et al.  Power systems stability control: reinforcement learning framework , 2004, IEEE Transactions on Power Systems.

[25]  Chongqing Kang,et al.  Carbon Emission Flow in Networks , 2012, Scientific Reports.

[26]  K. Nozaki,et al.  Feasibility of Distributed Carbon Capture and Storage (DCCS) , 2011 .

[27]  N.D. Hatziargyriou,et al.  Reinforcement learning for reactive power control , 2004, IEEE Transactions on Power Systems.

[28]  Serhat Duman,et al.  Optimal power flow using gravitational search algorithm , 2012 .

[29]  Enrico Blanzieri,et al.  Quantum Genetic Optimization , 2008, IEEE Transactions on Evolutionary Computation.

[30]  Mohammad Ali Abido,et al.  Multiobjective evolutionary algorithms for electric power dispatch problem , 2006, IEEE Transactions on Evolutionary Computation.

[31]  Xiaohua Xia,et al.  Multi-objective dynamic economic emission dispatch of electric power generation integrated with game theory based demand response programs , 2015 .

[32]  Chongqing Kang,et al.  Low-Carbon Power System Dispatch Incorporating Carbon Capture Power Plants , 2013, IEEE Transactions on Power Systems.

[33]  JiGuan G. Lin.  On min-norm and min-max methods of multi-objective optimization , 2005, Math. Program..

[34]  Kenji Iba.  Reactive power optimization by genetic algorithm , 1993 .

[35]  Chun-Lung Chen,et al.  Optimal power flow of a wind-thermal generation system , 2014 .

[36]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[37]  Jie Wu,et al.  Ranking approach of cross-efficiency based on improved TOPSIS technique , 2011 .

[38]  M. Basu,et al.  Multi-objective optimal power flow with FACTS devices , 2011 .

[39]  Zechun Hu,et al.  Carbon Flow Tracing Method for Assessment of Demand Side Carbon Emissions Obligation , 2013, IEEE Transactions on Sustainable Energy.

[40]  Sydulu Maheswarapu,et al.  Enhanced Genetic Algorithm based computation technique for multi-objective Optimal Power Flow solution , 2010 .