Data-driven decision making under uncertainty integrating robust optimization with principal component analysis and kernel smoothing methods

Abstract This paper proposes a novel data-driven robust optimization framework that leverages the power of machine learning and big data analytics for decision-making under uncertainty. By applying principal component analysis to uncertainty data, correlations between uncertain parameters are effectively captured, and latent uncertainty sources are identified. These data are then projected onto each principal component to facilitate extracting distributional information of latent uncertainties using kernel density estimation techniques. To explicitly account for asymmetric distributions, we introduce forward and backward deviation vectors into the data-driven uncertainty set, which are further incorporated into novel data-driven static and adaptive robust optimization models. The proposed framework not only significantly ameliorates the conservatism of robust optimization, but also enjoys computational efficiency and wide-ranging applicability. Three applications on optimization under uncertainty, including model predictive control, batch production scheduling, and process network planning, are presented to demonstrate the applicability of the proposed framework.

[1]  Rob J. Hyndman,et al.  A Bayesian approach to bandwidth selection for multivariate kernel density estimation , 2006, Comput. Stat. Data Anal..

[2]  Fengqi You,et al.  Unraveling Optimal Biomass Processing Routes from Bioconversion Product and Process Networks under Uncertainty: An Adaptive Robust Optimization Approach , 2016 .

[3]  R. Tütüncü,et al.  Adjustable Robust Optimization Models for a Nonlinear Two-Period System , 2008 .

[4]  I. Jolliffe.  Principal Component Analysis , 2002 .

[5]  D. Ruppert,et al.  Transformations in Regression: A Robust Analysis , 1985 .

[6]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[7]  Christodoulos A. Floudas,et al.  New a priori and a posteriori probabilistic bounds for robust counterpart optimization: I. Unknown probability distributions , 2016, Comput. Chem. Eng..

[8]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[9]  Michael Nikolaou,et al.  Chance‐constrained model predictive control , 1999 .

[10]  Yuan Yuan,et al.  Robust optimization under correlated uncertainty: Formulations and computational study , 2016, Comput. Chem. Eng..

[11]  Jorge Pinho de Sousa,et al.  Simultaneous regular and non-regular production scheduling of multipurpose batch plants: A real chemical-pharmaceutical case study , 2014, Comput. Chem. Eng..

[12]  F. Hampel.  The Influence Curve and Its Role in Robust Estimation , 1974 .

[13]  Richard E. Rosenthal,et al.  GAMS -- A User's Guide , 2004 .

[14]  David W. Scott,et al.  The Curse of Dimensionality and Dimension Reduction , 2008 .

[15]  Efstratios N. Pistikopoulos,et al.  Decomposition Based Stochastic Programming Approach for Polygeneration Energy Systems Design under Uncertainty , 2010 .

[16]  Ignacio E. Grossmann,et al.  Data-driven individual and joint chance-constrained optimization via kernel smoothing , 2015, Comput. Chem. Eng..

[17]  Qi Zhang,et al.  An adjustable robust optimization approach to scheduling of continuous industrial processes providing interruptible load , 2016, Comput. Chem. Eng..

[18]  Jean-Philippe Vial,et al.  Deriving robust counterparts of nonlinear uncertain inequalities , 2012, Math. Program..

[19]  Luis Puigjaner,et al.  Risk Management in the Scheduling of Batch Plants under Uncertain Market Demand , 2004 .

[20]  Efstratios N. Pistikopoulos,et al.  Uncertainty in process design and operations , 1995 .

[21]  Fengqi You,et al.  Efficient scheduling method of complex batch processes with general network structure via agent‐based modeling , 2013 .

[22]  Fengqi You,et al.  Data‐driven adaptive nested robust optimization: General modeling framework and efficient computational algorithm for decision making under uncertainty , 2017 .

[23]  Melvyn Sim,et al.  The Price of Robustness , 2004, Oper. Res..

[24]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[25]  E. Pistikopoulos,et al.  Proactive scheduling of batch processes by a combined robust optimization and multiparametric programming approach , 2013 .

[26]  Chao Shang,et al.  Data-driven robust optimization based on kernel learning , 2017, Comput. Chem. Eng..

[27]  Fengqi You,et al.  Adaptive robust optimization with minimax regret criterion: Multiobjective optimization framework and computational algorithm for planning and scheduling under uncertainty , 2018, Comput. Chem. Eng..

[28]  Chrysanthos E. Gounaris,et al.  Multi‐stage adjustable robust optimization for process scheduling under uncertainty , 2016 .

[29]  Basil Kouvaritakis,et al.  Model Predictive Control: Classical, Robust and Stochastic , 2015 .

[30]  Constantine Caramanis,et al.  Theory and Applications of Robust Optimization , 2010, SIAM Rev..

[31]  James B. Rawlings,et al.  Tutorial overview of model predictive control , 2000 .

[32]  Fengqi You,et al.  Stochastic inventory management for tactical process planning under uncertainties: MINLP models and algorithms , 2011 .

[33]  A. Barbosa‐Póvoa,et al.  Design and Scheduling of Periodic Multipurpose Batch Plants under Uncertainty , 2009 .

[34]  Chao Shang,et al.  Distributionally robust optimization for planning and scheduling under uncertainty , 2018, Comput. Chem. Eng..

[35]  Fengqi You,et al.  Optimal supply chain design and operations under multi-scale uncertainties: Nested stochastic robust optimization modeling framework and solution algorithm , 2016 .

[36]  Clayton D. Scott,et al.  Robust kernel density estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Manfred Morari,et al.  Model predictive control: Theory and practice - A survey , 1989, Autom..

[38]  Jonathan P. How,et al.  Bayesian nonparametric set construction for robust optimization , 2015, 2015 American Control Conference (ACC).

[39]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[40]  Fengqi You,et al.  A computational framework and solution algorithms for two‐stage adaptive robust scheduling of batch manufacturing processes under uncertainty , 2016 .

[41]  F. You,et al.  A data-driven multistage adaptive robust optimization framework for planning and scheduling under uncertainty , 2017 .

[42]  Allen L. Soyster,et al.  Technical Note - Convex Programming with Set-Inclusive Constraints and Applications to Inexact Linear Programming , 1973, Oper. Res..

[43]  Peng Sun,et al.  A Robust Optimization Perspective on Stochastic Programming , 2007, Oper. Res..

[44]  Efstratios N. Pistikopoulos,et al.  Stochastic optimization based algorithms for process synthesis under uncertainty , 1998 .

[45]  Efstratios N. Pistikopoulos,et al.  Model predictive control of anesthesia under uncertainty , 2014, Comput. Chem. Eng..

[46]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[47]  Dimitris Bertsimas,et al.  Duality in Two-Stage Adaptive Linear Optimization: Faster Computation and Stronger Bounds , 2016, INFORMS J. Comput..

[48]  Edrisi Muñoz,et al.  Scheduling and control decision-making under an integrated information environment , 2011, Comput. Chem. Eng..

[49]  Fengqi You,et al.  Data-Driven Stochastic Robust Optimization: General Computational Framework and Algorithm Leveraging Machine Learning for Optimization under Uncertainty in the Big Data Era , 2017, Comput. Chem. Eng..

[50]  Vishal Gupta,et al.  Data-driven robust optimization , 2013, Math. Program..

[51]  Fengqi You,et al.  Optimal processing network design under uncertainty for producing fuels and value‐added bioproducts from microalgae: Two‐stage adaptive robust mixed integer fractional programming model and computationally efficient solution algorithm , 2017 .