Max-affine estimators for convex stochastic programming

In this paper, we consider two sequential decision making problems with a convexity structure, namely an energy storage optimization task and a multi-product assembly example. We formulate these problems in the stochastic programming framework and discuss an approximate dynamic programming technique for their solutions. As the cost-to-go functions are convex in these cases, we use max-affine estimates for their approximations. To train such a max-affine estimate, we provide a new convex regression algorithm, and evaluate it empirically for these planning scenarios.

[1]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[2]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[3]  G. Dullerud.  A Computational Framework , 1996 .

[4]  Stephen P. Boyd,et al.  Convex piecewise-linear fitting , 2009 .

[5]  B. Sen,et al.  A Computational Framework for Multivariate Convex Regression and Its Variants , 2015, Journal of the American Statistical Association.

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  András György,et al.  Near-optimal max-affine estimators for convex regression , 2015, AISTATS.

[8]  Stein W. Wallace,et al.  Solving linear programs with multiple right-hand sides: Pricing and ordering schemes , 1996, Ann. Oper. Res..

[9]  Gene H. Golub,et al.  Matrix computations , 1983 .

[10]  David B. Dunson,et al.  Ensemble Methods for Convex Regression with Applications to Geometric Programming Based Circuit Design , 2012, ICML.

[11]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[12]  László Lovász,et al.  Hit-and-run mixes fast , 1999, Math. Program..

[13]  Peter Kall,et al.  Stochastic Programming , 1995 .

[14]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[15]  R. Wets,et al.  Stochastic programming , 1989 .

[16]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[17]  Antonio Alonso Ayuso,et al.  Introduction to Stochastic Programming , 2009 .

[18]  Warren B. Powell,et al.  An Approximate Dynamic Programming Algorithm for Monotone Value Functions , 2014, Oper. Res..

[19]  Robert L. Smith,et al.  Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions , 1984, Oper. Res..

[20]  David B. Dunson,et al.  Multivariate convex regression with adaptive partitioning , 2011, J. Mach. Learn. Res..

[21]  Johannes Bisschop,et al.  AIMMS - Optimization Modeling , 2006 .

[22]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.