Real time optimal feeding in a fermentor using a Markov decision algorithm

A Markov chain process is used to model a fed-batch fermentation. A real time optimization problem is posed in terms of Markov decision process (MDP) to determine the feed profile that maximizes the expected total discounted reward of the fermentation product. Preliminary optimality conditions, computational aspects, and real time experimental results are presented. The experimental results show a 120% increase in ethanol productivity compared with previous results.

[1]  L. Ingram,et al.  Effects of environmental conditions on xylose fermentation by recombinant Escherichia coli , 1990, Applied and environmental microbiology.

[2]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[3]  G. Zacchi,et al.  Kinetics of ethanol production by recombinant Escherichia coli KO11 , 1995, Biotechnology and bioengineering.

[4]  T. Park,et al.  Optimization of fermentation processes using recombinant Escherichia coli with the cloned trp operon , 1989, Biotechnology and bioengineering.

[5]  D. J. White,et al.  A Survey of Applications of Markov Decision Processes , 1993 .

[6]  G. T. Timmer,et al.  Stochastic global optimization methods part I: Clustering methods , 1987, Math. Program..

[7]  G Stephanopoulos,et al.  Optimization of fed‐batch penicillin fermentation: A case of singular optimal control with state constraints , 1989, Biotechnology and bioengineering.

[8]  Peter J. Kolesar,et al.  Optimal Dispatching of an Infinite-Capacity Shuttle: Control at a Single Terminal , 1974, Oper. Res..

[9]  Henk Tijms,et al.  A Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information , 1985 .

[10]  Bruce E. Stuckman,et al.  A comparison of Bayesian/sampling global optimization techniques , 1992, IEEE Trans. Syst. Man Cybern..

[11]  James C. Linden,et al.  COMPARISON OF ETHANOL PRODUCTION FROM XYLOSE BY A RECOMBINANT ESCHERICHIA COLI IN BATCH, FEDBATCH AND CONTINUOUS FERMENTATIONS , 1994 .

[12]  R. Mendelssohn.  Optimal harvesting strategies for stochastic single-species, multiage class models☆ , 1978 .

[13]  Lakhbir S. Hayre.  A note on optimal maintenance policies for deciding whether to repair or replace , 1983 .

[14]  Henk Tijms,et al.  Stochastic modelling and analysis: a computational approach , 1986 .

[15]  Roger J.-B. Wets,et al.  Minimization by Random Search Techniques , 1981, Math. Oper. Res..

[16]  L. Ingram,et al.  Expression of Different Levels of Ethanologenic Enzymes from Zymomonas mobilis in Recombinant Strains of Escherichia coli , 1988, Applied and environmental microbiology.

[17]  A Diener,et al.  Suboptimal control of fed-batch bioprocesses using phase properties. , 1994, Journal of biotechnology.

[18]  S. Enfors,et al.  Modeling of high cell density fed batch cultivation. , 1994, FEMS microbiology reviews.

[19]  Bruce E. Stuckman,et al.  A global search method for optimizing nonlinear systems , 1988, IEEE Trans. Syst. Man Cybern..

[20]  R. Bellman.  Dynamic programming. , 1957, Science.

[21]  D. J. White,et al.  Real Applications of Markov Decision Processes , 1985 .

[22]  A K Hilaly,et al.  Optimization of an industrial microalgae fermentation , 1994, Biotechnology and bioengineering.

[23]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[24]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[25]  Ferenc Szidarovszky,et al.  Linear Systems Theory , 1992 .

[26]  H C Lim,et al.  General characteristics of optimal feed rate profiles for various fed‐batch fermentation processes , 1986, Biotechnology and bioengineering.

[27]  Lyn C. Thomas,et al.  The blast furnaces problem , 1980 .

[28]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .