Reinforcement learning-based control for combined infusion of sedatives and analgesics

The focus of several clinical trials and research in the area of clinical pharmacology is to fine tune the drug dosing in the phase of additive, antagonistic, and synergistic drug interactive effects. It is important to consider the interactive effects of the drugs to restrict the drug usage to the optimal level required to achieve certain therapeutic effects. Such optimal drug dosing methods will minimize the adverse drug effects and cost associated with the treatment. In this paper, we discuss the use of a reinforcement learning (RL)-based controller to fine tune the drug titration while different drugs with interactive effects are administrated simultaneously. We demonstrate the efficacy of the method by using 25 simulated patients for the simultaneous infusion of a sedative and analgesic drug which has synergistic interactive effect.

[1]  K T Muir,et al.  Pharmacokinetics and Pharmacodynamics of Remifentanil in Volunteer Subjects with Severe Liver Disease , 1996, Anesthesiology.

[2]  J. Beleña,et al.  Randomized double-blind comparison of remifentanil and alfentanil in patients undergoing laparoscopic cholecystectomy using total intravenous anesthesia , 2016, Journal of anaesthesiology, clinical pharmacology.

[3]  Steven L Shafer,et al.  Is Synergy the Rule? A Review of Anesthetic Interactions Producing Hypnosis and Immobility , 2008, Anesthesia and analgesia.

[4]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  T. Fuhrman,et al.  Unexpected behaviour of the bispectral index (BIS) after brain injury , 2012, Anaesthesia.

[7]  Larry D. Pyeatt,et al.  Reinforcement learning for closed-loop propofol anesthesia: a study in human volunteers , 2014, J. Mach. Learn. Res..

[8]  M. Jann,et al.  Clinically Significant Interactions with Anesthetic Agents , 2016 .

[9]  Wassim M. Haddad,et al.  Closed-loop control of anesthesia and mean arterial pressure using reinforcement learning , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[10]  Sangeeta Mehta,et al.  Canadian survey of the use of sedatives, analgesics, and neuromuscular blocking agents in critically ill patients* , 2006, Critical care medicine.

[11]  R. Raffa,et al.  What do we (not) know about how paracetamol (acetaminophen) works? , 2010, Journal of clinical pharmacy and therapeutics.

[12]  Wassim M. Haddad,et al.  Optimal drug dosing control for intensive care unit sedation by using a hybrid deterministic–stochastic pharmacokinetic and pharmacodynamic model , 2013 .

[13]  W. Haddad,et al.  Nonnegative and Compartmental Dynamical Systems , 2010 .

[14]  S L Shafer,et al.  Response Surface Model for Anesthetic Drug Interactions , 2000, Anesthesiology.

[15]  Philip D Lumb,et al.  Clinical practice guidelines for the sustained use of sedatives and analgesics in the critically ill adult. , 2002, Critical care medicine.

[16]  Ulf Bodin,et al.  Machine learning approach to automatic bucket loading , 2016, 2016 24th Mediterranean Conference on Control and Automation (MED).

[17]  Wassim M. Haddad,et al.  Direct adaptive disturbance rejection control for sedation and analgesia , 2014, 2nd Middle East Conference on Biomedical Engineering.

[18]  Zhi-Quan Luo,et al.  A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing , 2015, IEEE Signal Processing Magazine.