Scalable stochastic optimization of complex energy systems

We present a scalable approach and implementation for solving stochastic programming problems, with application to the optimization of complex energy systems under uncertainty. Stochastic programming is used to make decisions in the present while incorporating a model of uncertainty about future events (scenarios). These problems present serious computational difficulties as the number of scenarios becomes large and the complexity of the system and planning horizons increase, necessitating the use of parallel computing. Our novel hybrid parallel implementation PIPS is based on interior-point methods and uses a Schur complement technique to obtain a scenario-based decomposition of the linear algebra. PIPS is applied to a stochastic economic dispatch problem that uses hourly wind forecasts and a detailed physical power flow model. Solving this problem is necessary for efficient integration of wind power with the Illinois power grid and real-time energy market. Strong scaling efficiency of 96% is obtained on 32 racks (131,072 cores) of the "Intrepid" Blue Gene/P system at Argonne National Laboratory.

[1]  Robert A. van de Geijn,et al.  Elemental: A New Framework for Distributed Memory Dense Matrix Computations , 2013, TOMS.

[2]  Patrick Amestoy,et al.  Hybrid scheduling for the parallel solution of linear systems , 2006, Parallel Comput..

[3]  J. Bunch,et al.  Direct Methods for Solving Symmetric Indefinite Systems of Linear Equations , 1971 .

[4]  Stephen J. Wright,et al.  Decomposition Algorithms for Stochastic Programming on a Computational Grid , 2001, Comput. Optim. Appl..

[5]  Abhishek Kumar Gupta,et al.  Wsmp: watson sparse matrix package , 2000 .

[6]  Mihai Anitescu,et al.  A preconditioning technique for Schur complement systems arising in stochastic optimization , 2012, Comput. Optim. Appl..

[7]  Zuyi Li,et al.  Market Operations in Electric Power Systems : Forecasting, Scheduling, and Risk Management , 2002 .

[8]  Jacek Gondzio,et al.  Parallel interior-point solver for structured linear programs , 2003, Math. Program..

[9]  Mihai Anitescu,et al.  The parallel solution of dense saddle-point linear systems arising in stochastic programming , 2012, Optim. Methods Softw..

[10]  Victor M. Zavala,et al.  Computational and economic limitations of dispatch operations in the next-generation power grid , 2010, 2010 IEEE Conference on Innovative Technologies for an Efficient and Reliable Electricity Supply.

[11]  Sanjay Mehrotra,et al.  Decomposition Based Interior Point Methods for Two-Stage Stochastic Convex Quadratic Programs with Recourse , 2009, Oper. Res..

[12]  Jacek Gondzio,et al.  Exploiting structure in parallel implementation of interior point methods for optimization , 2009, Comput. Manag. Sci..

[13]  Patrick Amestoy,et al.  A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling , 2001, SIAM J. Matrix Anal. Appl..

[14]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[15]  O. Schenk,et al.  ON FAST FACTORIZATION PIVOTING METHODS FOR SPARSE SYMMETRI C INDEFINITE SYSTEMS , 2006 .

[16]  Jack Dongarra,et al.  ScaLAPACK Users' Guide , 1987 .

[17]  J. Birge,et al.  Computing block-angular Karmarkar projections with applications to stochastic programming , 1988 .

[18]  Jacek Gondzio,et al.  Parallel interior-point solver for structured quadratic programs: Application to financial planning problems , 2007, Ann. Oper. Res..

[19]  Antonio Alonso Ayuso,et al.  Introduction to Stochastic Programming , 2009 .

[20]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[21]  Jean-Yves L'Excellent,et al.  Some Experiments and Issues to Exploit Multicore Parallelism in a Distributed-Memory Parallel Sparse Direct Solver , 2010 .

[22]  Jacek Gondzio,et al.  Direct Solution of Linear Systems of Size 109 Arising in Optimization with Interior Point Methods , 2005, PPAM.

[23]  Victor M. Zavala,et al.  Interior-point decomposition approaches for parallel solution of large-scale nonlinear parameter estimation problems , 2008 .

[24]  Sangmin Lee,et al.  A Computational Framework for Uncertainty Quantification and Stochastic Optimization in Unit Commitment With Wind Power Generation , 2011, IEEE Transactions on Power Systems.

[25]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .