Tutorial on Stochastic Optimization in Energy—Part I: Modeling and Policies

There is a wide range of problems in energy systems that require making decisions in the presence of different forms of uncertainty. The fields that address sequential, stochastic decision problems lack a standard canonical modeling framework, with fragmented, competing solution strategies. Recognizing that we will never agree on a single notational system, this two-part tutorial proposes a simple, straightforward canonical model (that is most familiar to people with a control theory background), and introduces four fundamental classes of policies which integrate the competing strategies that have been proposed under names such as control theory, dynamic programming, stochastic programming and robust optimization. Part II of the tutorial illustrates the modeling framework using a simple energy storage problem, where we show that, depending on the problem characteristics, each of the four classes of policies may be best.

[1]  Dimitris Bertsimas,et al.  Tractable stochastic analysis in high dimensions via robust optimization , 2012, Math. Program..

[2]  Peter Kall,et al.  Stochastic Programming , 1995 .

[3]  R. Bellman Dynamic programming. , 1957, Science.

[4]  Michael C. Fu,et al.  Optimization for Simulation: Theory vs. Practice , 2002 .

[5]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[6]  Boaz Golany,et al.  Retailer-Supplier Flexible Commitments Contracts: A Robust Optimization Approach , 2005, Manuf. Serv. Oper. Manag..

[7]  N.P. Padhy,et al.  Unit commitment-a bibliographical survey , 2004, IEEE Transactions on Power Systems.

[8]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Vol. II , 1976 .

[9]  Jery R. Stedinger,et al.  SOCRATES: A system for scheduling hydroelectric generation under uncertainty , 1995, Ann. Oper. Res..

[10]  Xu Andy Sun,et al.  Adaptive Robust Optimization for the Security Constrained Unit Commitment Problem , 2013, IEEE Transactions on Power Systems.

[11]  John R. Birge,et al.  Introduction to Stochastic Programming , 1997 .

[12]  Michael C. Fu,et al.  Feature Article: Optimization for simulation: Theory vs. Practice , 2002, INFORMS J. Comput..

[13]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[14]  Warren B. Powell,et al.  A comparison of approximate dynamic programming techniques on benchmark energy storage problems: Does anything work? , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[15]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[16]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[17]  Frank L. Lewis,et al.  Optimal Control: Lewis/Optimal Control 3e , 2012 .

[18]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[19]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[20]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[21]  Robert F. Stengel,et al.  Optimal Control and Estimation , 1994 .

[22]  Donald E. Kirk,et al.  Optimal control theory : an introduction , 1970 .

[23]  John R. Birge,et al.  A stochastic model for the unit commitment problem , 1996 .

[24]  Andrzej Ruszczynski,et al.  Risk-averse dynamic programming for Markov decision processes , 2010, Math. Program..

[25]  R. Stengel Stochastic Optimal Control: Theory and Application , 1986 .

[26]  John R. Birge,et al.  Introduction to Stochastic programming (2nd edition), Springer verlag, New York , 2011 .

[27]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[28]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[29]  David L. Woodruff,et al.  Toward scalable, parallel progressive hedging for stochastic unit commitment , 2013, 2013 IEEE Power & Energy Society General Meeting.

[30]  M. V. F. Pereira,et al.  Multi-stage stochastic optimization applied to energy planning , 1991, Math. Program..

[31]  Nicola Secomandi,et al.  An Approximate Dynamic Programming Approach to Benchmark Practice-Based Heuristics for Natural Gas Storage Valuation , 2010, Oper. Res..

[32]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[33]  Melvyn Sim,et al.  Distributionally Robust Optimization and Its Tractable Approximations , 2010, Oper. Res..

[34]  Warren B. Powell,et al.  Clearing the Jungle of Stochastic Optimization , 2014 .

[35]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[36]  R. Rockafellar,et al.  The fundamental risk quadrangle in risk management, optimization and statistical estimation , 2013 .

[37]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[38]  Daniel Kuhn,et al.  Distributionally Robust Convex Optimization , 2014, Oper. Res..

[39]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[40]  Vincenzo Marano,et al.  A stochastic dynamic programming model for co-optimization of distributed energy storage , 2013, Energy Systems.

[41]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[42]  Eduardo D. Sontag,et al.  Mathematical control theory: deterministic finite dimensional systems (2nd ed.) , 1998 .

[43]  J. M. Arroyo,et al.  Contingency-Constrained Unit Commitment With $n - K$ Security Criterion: A Robust Optimization Approach , 2011, IEEE Transactions on Power Systems.

[44]  Alexander Shapiro,et al.  Minimax and risk averse multistage stochastic programming , 2012, Eur. J. Oper. Res..

[45]  Bernhard Sendhoff,et al.  Robust Optimization - A Comprehensive Survey , 2007 .

[46]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .