Dynamic programming approach to the numerical solution of optimal control with paradigm by a mathematical model for drug therapies of HIV/AIDS

In this paper, we present a new numerical algorithm to find the optimal control for the general nonlinear lumped systems without state constraints. The dynamic programming-viscosity solution (DPVS) approach is developed and the numerical solutions of both approximate optimal control and trajectory are produced. To show the effectiveness and efficiency of new algorithm, we apply it to an optimal control problem of two types of drug therapies for human immunodeficiency virus (HIV)/acquired immune deficiency syndrome (AIDS). The quality of the obtained optimal control and the trajectory pair is checked through comparison with the costs under the arbitrarily selected different controls. The results illustrate the effectiveness of the algorithm.

[1]  Glenn F. Webb,et al.  Immunotherapy of HIV-1 Infection , 1998 .

[2]  Harvey Thomas Banks,et al.  Receding Horizon Control of HIV , 2011 .

[3]  R. Sargent.  Optimal control , 2000 .

[4]  M. Nowak,et al.  Virus dynamics: Mathematical principles of immunology and virology , 2001 .

[5]  B. Adams,et al.  Dynamic multidrug therapies for hiv: optimal and sti control approaches. , 2004, Mathematical biosciences and engineering : MBE.

[6]  Oskar von Stryk,et al.  Direct and indirect methods for trajectory optimization , 1992, Ann. Oper. Res..

[7]  J. Arora,et al.  Differential dynamic programming technique for optimal control , 1994 .

[8]  Libin Rong,et al.  Modeling HIV persistence, the latent reservoir, and viral blips. , 2009, Journal of theoretical biology.

[9]  Bing Sun,et al.  Numerical solution to the optimal feedback control of continuous casting process , 2007, J. Glob. Optim..

[10]  D. Mayne,et al.  First-order strong variation algorithms for optimal control , 1975 .

[11]  D. Pan,et al.  Upwind finite-volume method for natural and forced convection , 1994 .

[12]  Piero Lanucara,et al.  A splitting algorithm for Hamilton-Jacobi-Bellman equations , 1992 .

[13]  M. Nowak,et al.  Dynamic multidrug therapies for HIV: a control theoretic approach. , 2015, Journal of theoretical biology.

[14]  Xiaohua Xia,et al.  When to initiate HIV therapy: a control theoretic approach , 2003, IEEE Transactions on Biomedical Engineering.

[15]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[16]  J.A.M. Felippe de Souza,et al.  Optimal control theory applied to the anti-viral treatment of AIDS , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[17]  P. Sellier,et al.  Interleukin-2 therapy in patients with HIV infection. , 2010, The New England journal of medicine.

[18]  Guanrong Chen,et al.  Feedback control of a biodynamical model of HIV-1 , 2001, IEEE Transactions on Biomedical Engineering.

[19]  K. Teo,et al.  On application of an alternating direction method to Hamilton-Jacobin-Bellman equations , 2004 .

[20]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[21]  Yu-Chi Ho.  On centralized optimal control , 2005, IEEE Transactions on Automatic Control.

[22]  John T. Workman,et al.  Optimal Control Applied to Biological Models , 2007 .

[23]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[24]  B. Guo,et al.  Numerical solution to the optimal birth feedback control of a population dynamics: viscosity solution approach , 2005 .

[25]  Kok Lay Teo,et al.  Numerical Solution of Hamilton-Jacobi-Bellman Equations by an Upwind Finite Volume Method , 2003, J. Glob. Optim..

[26]  H T Banks,et al.  Estimation and Prediction With HIV-Treatment Interruption Data , 2007, Bulletin of mathematical biology.

[27]  P. Lions,et al.  Two approximations of solutions of Hamilton-Jacobi equations , 1984 .

[28]  Alan S. Perelson,et al.  Emergence of HIV-1 Drug Resistance During Antiretroviral Treatment , 2007, Bulletin of mathematical biology.

[29]  Ryan Zurakowski,et al.  A model predictive control based scheduling method for HIV therapy. , 2006, Journal of theoretical biology.

[30]  K. Teo,et al.  Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristics , 2000 .

[31]  Xiaohua Xia,et al.  Introducing HIV/AIDS education into the electrical engineering curriculum at the University of Pretoria , 2004, IEEE Transactions on Education.

[32]  Wolfgang Hackbusch,et al.  A numerical method for solving parabolic equations with opposite orientations , 1978, Computing.

[33]  Song Wang,et al.  Numerical solution of Hamilton–Jacobi–Bellman equations by an exponentially fitted finite volume method , 2006 .

[34]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[35]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[36]  J. Stoer,et al.  Introduction to Numerical Analysis , 2002 .

[37]  B. Adams,et al.  HIV dynamics: Modeling, data analysis, and optimal treatment protocols , 2005 .

[38]  J. A. Bryson.  Optimal control-1950 to 1985 , 1996 .

[39]  Hem Raj Joshi,et al.  Optimal control of an HIV immunology model , 2002 .

[40]  Kok Lay Teo,et al.  An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations , 2000 .

[41]  Rafael Castro-Linares,et al.  Trajectory tracking for non-holonomic cars: A linear approach to controlled leader-follower formation , 2010, 49th IEEE Conference on Decision and Control (CDC).

[42]  Yongsheng Ding,et al.  Optimal Control of a Fractional-Order HIV-Immune System With Memory , 2012, IEEE Transactions on Control Systems Technology.

[43]  Verica Radisavljevic-Gajic,et al.  Optimal Control of HIV-Virus Dynamics , 2009, Annals of Biomedical Engineering.

[44]  Denise E. Kirschner,et al.  Using Mathematics to Understand HIV Immune Dynamics , 1997 .

[45]  D. Kirschner,et al.  Optimal control of the chemotherapy of HIV , 1997, Journal of mathematical biology.

[46]  Alan S. Perelson,et al.  Mathematical Analysis of HIV-1 Dynamics in Vivo , 1999, SIAM Rev..

[47]  I. Craig,et al.  Can HIV/AIDS be controlled? Applying control engineering concepts outside traditional fields , 2005, IEEE Control Systems.

[48]  Robert F. Stengel,et al.  Optimal enhancement of immune response , 2002, Bioinform..

[49]  G. Leitmann,et al.  Mathematical Methods of Optimal Control , 1971 .

[50]  Ralf Blossey,et al.  Computational Biology (Chapman & Hall/Crc Mathematical and Computational Biology Series) , 2006 .