Retrospective on optimization

In this paper, we provide a general classification of mathematical optimization problems, followed by a matrix of applications that shows the areas in which these problems have been typically applied in process systems engineering. We then provide a review of solution methods of the major types of optimization problems for continuous and discrete variable optimization, particularly nonlinear and mixed-integer nonlinear programming (MINLP). We also review their extensions to dynamic optimization and optimization under uncertainty. While these areas are still subject to significant research efforts, the emphasis in this paper is on major developments that have taken place over the last 25 years.

[1]  Ignacio E. Grossmann,et al.  Mathematical programming approaches to the synthesis of chemical process systems , 1999 .

[2]  M. L. Chambers.  The Mathematical Theory of Optimal Processes , 1965 .

[3]  Sanjay Mehrotra,et al.  A branch-and-cut method for 0-1 mixed convex programming , 1999, Math. Program..

[4]  D. Himmelblau,et al.  Optimization of Chemical Processes , 1987 .

[5]  Dafydd Gibbon,et al.  1 User’s guide , 1998 .

[6]  A. M. Geoffrion.  Generalized Benders decomposition , 1972 .

[7]  A. Ruszczynski.  Stochastic Programming Models , 2003 .

[8]  Serge Domenech,et al.  Separation sequence synthesis how to use simulated annealing procedure , 1993 .

[9]  Efstratios N. Pistikopoulos,et al.  Optimal retrofit design for improving process flexibility in nonlinear systems—II. Optimal level of flexibility , 1989 .

[10]  Horand I. Gassmann,et al.  Mslip: A computer code for the multistage stochastic linear programming problem , 1990, Math. Program..

[11]  John R. Birge,et al.  Decomposition and Partitioning Methods for Multistage Stochastic Linear Programs , 1985, Oper. Res..

[12]  John E. Mitchell,et al.  An improved branch and bound algorithm for mixed integer nonlinear programs , 1994, Comput. Oper. Res..

[13]  Robert J. Vanderbei,et al.  An Interior-Point Algorithm for Nonconvex Nonlinear Programming , 1999, Comput. Optim. Appl..

[14]  L. Biegler,et al.  Decomposition algorithms for on-line estimation with nonlinear DAE models , 1995 .

[15]  Ignacio E. Grossmann,et al.  Modeling uncertainty and analyzing bottleneck characteristics in multiperiod design optimization , 1995 .

[16]  John R. Birge,et al.  Stochastic Programming Computation and Applications , 1997, INFORMS J. Comput..

[17]  Efstratios N. Pistikopoulos,et al.  Optimal retrofit design for improving process flexibility in linear systems , 1988 .

[18]  R. Luus,et al.  Optimal Control of Inequality State Constrained Systems , 1997 .

[19]  Thomas L. Magnanti,et al.  Accelerating Benders Decomposition: Algorithmic Enhancement and Model Selection Criteria , 1981, Oper. Res..

[20]  Ignacio E. Grossmann,et al.  Mixed-Integer Optimization Techniques for Algorithmic Process Synthesis , 1996 .

[21]  Ignacio E. Grossmann,et al.  Mixed-Integer Nonlinear Programming: A Survey of Algorithms and Applications , 1997 .

[22]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[23]  Hans Georg Bock,et al.  A Boundary Value Problem Approach to the Optimizationof Chemical Processes Described , 1997 .

[24]  Christodoulos A. Floudas,et al.  Process Synthesis, Design, and Control: A Mixed-Integer Optimal Control Framework , 1998 .

[25]  Rein Luus,et al.  Use of random admissible values for control in iterative dynamic programming , 1992 .

[26]  D. K. Varvarezos,et al.  Multiperiod design optimization with SQP decomposition , 1994 .

[27]  Yuri Ermoliev,et al.  Numerical techniques for stochastic optimization , 1988 .

[28]  Marianthi G. Ierapetritou,et al.  New Approach for Quantifying Process Feasibility: Convex and 1-D Quasi-Convex Regions , 2001 .

[29]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .

[30]  Sven Leyffer,et al.  Deterministic Methods for Mixed Integer Nonlinear Programming , 1993 .

[31]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[32]  Nilay Shah,et al.  Modelling and optimisation of general hybrid systems in the continuous time domain , 1998 .

[33]  Sophia Lefantzi,et al.  DAKOTA : a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. , 2011 .

[34]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[35]  Paul I. Barton,et al.  Dynamic Optimization in a Discontinuous World , 1998 .

[36]  I. Grossmann,et al.  Convergence properties of generalized benders decomposition , 1991 .

[37]  A. Madansky.  Inequalities for Stochastic Linear Programming Problems , 1960 .

[38]  Virginia Torczon,et al.  On the Convergence of the Multidirectional Search Algorithm , 1991, SIAM J. Optim..

[39]  G. M. Ostrovsky,et al.  An approach to solving a two-stage optimization problem under uncertainty , 1997 .

[40]  J. Dennis,et al.  Direct Search Methods on Parallel Machines , 1991 .

[41]  Ignacio E. Grossmann,et al.  Optimal process design under uncertainty , 1983 .

[42]  Jorge Nocedal,et al.  An Interior Point Algorithm for Large-Scale Nonlinear Programming , 1999, SIAM J. Optim..

[43]  R. Goulcher,et al.  The solution of steady-state chemical engineering optimisation problems using a random-search algorithm , 1978 .

[44]  Nicholas I. M. Gould,et al.  Global Convergence of a Trust-Region SQP-Filter Algorithm for General Nonlinear Programming , 2002, SIAM J. Optim..

[45]  George H. Staus,et al.  Interior point SQP strategies for large-scale, structured process optimization problems , 1999 .

[46]  L. Biegler,et al.  Dynamic Optimization in the Design and Scheduling of Multiproduct Batch Plants , 1996 .

[47]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[48]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[49]  P. I. Barton,et al.  Dynamic optimization with state variable path constraints , 1998 .

[50]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[51]  Michael A. Saunders,et al.  USER’S GUIDE FOR SNOPT 5.3: A FORTRAN PACKAGE FOR LARGE-SCALE NONLINEAR PROGRAMMING , 2002 .

[52]  Eric S. Fraga,et al.  Mass exchange network synthesis using genetic algorithms , 1998 .

[53]  L. Biegler,et al.  Large‐scale DAE optimization using a simultaneous NLP formulation , 1998 .

[54]  William T. Ziemba,et al.  11. Introduction to Stochastic Programming Applications , 2005, Applications of Stochastic Programming.

[55]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[56]  Sebastian Engell,et al.  Sequencing of batch operations for a highly coupled production process: Genetic algorithms versus mathematical programming , 1998 .

[57]  Kalyanmoy Deb,et al.  Sensor network design of linear processes using genetic algorithms , 1998 .

[58]  L. Biegler,et al.  Advances in simultaneous strategies for dynamic process optimization , 2002 .

[59]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[60]  Ignacio E. Grossmann,et al.  An outer-approximation method for multiperiod design optimization , 1992 .

[61]  Ignacio E. Grossmann,et al.  A sensitivity based approach for flexibility analysis and design of linear process systems , 1995 .

[62]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[63]  Lorenz T. Biegler,et al.  Global and Local Convergence of Line Search Filter Methods for Nonlinear Programming , 2002 .

[64]  Stephen J. Wright,et al.  Application of Interior-Point Methods to Model Predictive Control , 1998 .

[65]  Philip E. Gill,et al.  Practical optimization , 1981 .

[66]  Egon Balas,et al.  A lift-and-project cutting plane algorithm for mixed 0–1 programs , 1993, Math. Program..

[67]  Paul I. Barton,et al.  Mixed-integer dynamic optimization , 1997 .

[68]  L. Hasdorff.  Gradient Optimization and Nonlinear Control , 1976 .

[69]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[70]  R. Luus.  Piecewise linear continuous optimal control by iterative dynamic programming , 1993 .

[71]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[72]  U. Ascher,et al.  A New Basis Implementation for a Mixed Order Boundary Value ODE Solver , 1987 .

[73]  Ignacio E. Grossmann,et al.  An index for operational flexibility in chemical process design. Part I: Formulation and theory , 1985 .

[74]  Paul I. Barton,et al.  Generalized branch-and-cut framework for mixed-integer nonlinear optimization problems , 2000 .

[75]  X. Yuan,et al.  Une méthode d'optimisation non linéaire en variables mixtes pour la conception de procédés , 1988 .

[76]  In-Beum Lee,et al.  A genetic algorithm for scheduling of multi-product batch processes , 1998 .

[77]  Efstratios N. Pistikopoulos,et al.  Simultaneous incorporation of flexibility and economic risk in operational planning under uncertainty , 1994 .

[78]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[79]  R. J. Dakin,et al.  A tree-search algorithm for mixed integer programming problems , 1965, Comput. J..

[80]  G. R. Sullivan,et al.  Development of feed changeover policies for refinery distillation units , 1979 .

[81]  Ignacio E. Grossmann,et al.  An index for operational flexibility in chemical process design. Part I , 1983 .

[82]  Nikolaos V. Sahinidis,et al.  BARON: A general purpose global optimization software package , 1996, J. Glob. Optim..

[83]  Sven Leyffer,et al.  Integrating SQP and Branch-and-Bound for Mixed Integer Nonlinear Programming , 2001, Comput. Optim. Appl..

[84]  Paul I. Barton,et al.  Mixed-integer dynamic optimization I: problem formulation , 1999 .

[85]  R. Weiner.  Lecture Notes in Economics and Mathematical Systems , 1985 .

[86]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[87]  Efstratios N. Pistikopoulos,et al.  A novel flexibility analysis approach for processes with stochastic parameters , 1990 .

[88]  Ignacio E. Grossmann,et al.  Assignment and sequencing models for thescheduling of process systems , 1998, Ann. Oper. Res..

[89]  Lorenz T. Biegler,et al.  Incorporating joint confidence regions into design under uncertainty , 1999 .

[90]  R. Sargent,et al.  Optimum Design of Multipurpose Chemical Plants , 1979 .

[91]  Lorenz T. Biegler,et al.  Failure of global convergence for a class of interior point methods for nonlinear programming , 2000, Math. Program..

[92]  I. Grossmann,et al.  A combined penalty function and outer-approximation method for MINLP optimization : applications to distillation column design , 1989 .

[93]  J. E. Cuthrell,et al.  Simultaneous optimization and solution methods for batch reactor control profiles , 1989 .

[94]  S. A. Da Eeo,et al.  DYNAMIC OPTIMIZATION OF CONSTRAINED CHEMICAL ENGINEERING PROBLEMS USING DYNAMIC PROGRAMMING , 1995 .

[95]  Omprakash K. Gupta,et al.  Branch and Bound Experiments in Convex Nonlinear Integer Programming , 1985 .

[96]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[97]  Josef Kallrath,et al.  Mixed Integer Optimization in the Chemical Process Industry: Experience, Potential and Future Perspectives , 2000 .

[98]  Edward M. B. Smith,et al.  A symbolic reformulation/spatial branch-and-bound algorithm for the global optimisation of nonconvex MINLPs , 1999 .

[99]  Katya Scheinberg,et al.  Recent progress in unconstrained nonlinear optimization without derivatives , 1997, Math. Program..

[100]  L. N. Vicente,et al.  Trust-Region Interior-Point SQP Algorithms for a Class of Nonlinear Programming Problems , 1998 .

[101]  Ignacio E. Grossmann,et al.  Optimization strategies for flexible chemical processes , 1983 .

[102]  M. S. Bazaraa,et al.  Nonlinear Programming , 1979 .

[103]  J. E. Cuthrell,et al.  On the optimization of differential-algebraic process systems , 1987 .

[104]  Martin W. P. Savelsbergh,et al.  Branch-and-Price: Column Generation for Solving Huge Integer Programs , 1998, Oper. Res..

[105]  Martin W. P. Savelsbergh,et al.  Progress in Linear Programming-Based Algorithms for Integer Programming: An Exposition , 2000, INFORMS J. Comput..

[106]  Arthur E. Bryson,et al.  Applied Optimal Control , 1969 .

[107]  Alexander H. G. Rinnooy Kan,et al.  Decomposition in general mathematical programming , 1993, Math. Program..

[108]  Martin Grötschel,et al.  Online optimization of large scale systems , 2001 .

[109]  Nikolaos V. Sahinidis,et al.  Optimization under uncertainty: state-of-the-art and opportunities , 2004, Comput. Chem. Eng..

[110]  Frederick S. Hillier,et al.  Introduction of Operations Research , 1967 .

[111]  George B. Dantzig,et al.  Linear Programming Under Uncertainty , 2004, Manag. Sci..

[112]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[113]  Lorenz T. Biegler,et al.  Synthesis of Optimal Chemical Reactor Networks , 1996 .

[114]  Wolfgang Dahmen,et al.  Introduction to Model Based Optimization of Chemical Processes on Moving Horizons , 2001 .

[115]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[116]  Tapio Westerlund,et al.  A cutting plane method for minimizing pseudo-convex functions in the mixed integer case , 2000 .

[117]  Jacques F. Benders,et al.  Partitioning procedures for solving mixed-variables programming problems , 2005, Comput. Manag. Sci..

[118]  Ignacio E. Grossmann,et al.  Optimum design of chemical plants with uncertain parameters , 1978 .

[119]  S. Nilchan,et al.  On the Optimisation of Periodic Adsorption Processes , 1998 .

[120]  François V. Louveaux,et al.  Multistage stochastic programs with block-separable recourse , 1986 .

[121]  Jeremy F. Shapiro,et al.  Mathematical programming models and methods for production planning and scheduling , 1988 .

[122]  R. K. Malik,et al.  Optimal design of flexible chemical processes , 1979 .

[123]  H. Das,et al.  Scheduling of serial multiproduct batch processes via simulated annealing , 1990 .

[124]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[125]  Julio R. Banga,et al.  Global Optimization of Chemical Processes using Stochastic Algorithms , 1996 .

[126]  Rein Luus,et al.  A direct approach to optimization of a complex system , 1973 .

[127]  R. Sargent,et al.  Solution of a Class of Multistage Dynamic Optimization Problems. 2. Problems with Path Constraints , 1994 .

[128]  John R. Birge,et al.  Introduction to Stochastic Programming , 1997 .

[129]  Ignacio E. Grossmann,et al.  Systematic Methods of Chemical Process Design , 1997 .

[130]  George E. P. Box,et al.  Evolutionary Operation: a Method for Increasing Industrial Productivity , 1957 .

[131]  Ignacio E. Grossmann,et al.  Mixed-Integer Optimization Techniques for the Design and Scheduling of Batch Processes , 1996 .

[132]  A. C. Hoffmann,et al.  AIChE Symposium Series , 1999 .

[133]  R. Wets,et al.  Stochastic programming , 1989 .

[134]  David Kendrick,et al.  GAMS, a user's guide , 1988, SGNM.

[135]  Lorenz T. Biegler,et al.  Dynamic Process Optimization through Adjoint Formulations and Constraint Aggregation , 1999 .

[136]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[137]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[138]  G. Reddien.  Collocation at Gauss Points as a Discretization in Optimal Control , 1979 .

[139]  J. A. Bather,et al.  Optimization of Stochastic Systems: Topics in Discrete-Time Dynamics , 1989 .

[140]  Behram Jehanbux Hansotia,et al.  Stochastic Programming , 1995, Optimizations and Programming.

[141]  Ignacio E. Grossmann,et al.  An outer-approximation algorithm for a class of mixed-integer nonlinear programs , 1987, Math. Program..

[142]  Vassilios Vassiliadis,et al.  Computational solution of dynamic optimization problems with general differential-algebraic constraints , 1993 .

[143]  Robert E. Bixby,et al.  MIP: Theory and Practice - Closing the Gap , 1999, System Modelling and Optimization.

[144]  Sven Leyffer,et al.  Solving mixed integer nonlinear programs by outer approximation , 1994, Math. Program..

[145]  Elijah Polak,et al.  Optimization: Algorithms and Consistent Approximations , 1997 .

[146]  I. Grossmann,et al.  Relaxation strategy for the structural optimization of process flow sheets , 1987 .

[147]  L. Biegler,et al.  Large-scale dynamic optimization for grade transitions in a low density polyethylene plant , 2002 .

[148]  C. Floudas,et al.  Active constraint strategy for flexibility analysis in chemical processes , 1987 .

[149]  Peter T. Cummings,et al.  Process optimization via simulated annealing: Application to network design , 1989 .

[150]  I. Grossmann,et al.  An LP/NLP based branch and bound algorithm for convex MINLP optimization problems , 1992 .

[151]  Ignacio E. Grossmann,et al.  Design optimization of stochastic flexibility , 1993 .

[152]  Ignacio E. Grossmann,et al.  Advances in Mathematical Programming for Automated Design , Integration and Operation of Chemical Processes , 1999 .

[153]  Jeffery S. Logsdon,et al.  Accurate solution of differential-algebraic optimization problems , 1989 .

[154]  Ignacio E. Grossmann,et al.  Recent Developments in the Evaluation and Optimization of Flexible Chemical Processes , 1996 .

[155]  L. Biegler,et al.  Stable Decomposition for Dynamic Optimization , 1995 .