Towards efficient, personalized anesthesia using continuous reinforcement learning for propofol infusion control

We demonstrate the use of reinforcement learning algorithms for efficient and personalized control of patients' depth of general anesthesia during surgical procedures - an important aspect for Neurotechnology. We used the continuous actor-critic learning automaton technique, which was trained and tested in silico using published patient data, physiological simulation and the bispectral index (BIS) of patient EEG. Our two-stage technique learns first a generic effective control strategy based on average patient data (factory stage) and can then fine-tune itself to individual patients (personalization stage). The results showed that the reinforcement learner as compared to a bang-bang controller reduced the dose of the anesthetic agent administered by 9.4% and kept the patient closer to the target state, as measured by RMSE (4.90 compared to 8.47). It also kept the BIS error within a narrow, clinically acceptable range 93.9% of the time. Moreover, the policy was trained using only 50 simulated operations. Being able to learn a control strategy this quickly indicates that the reinforcement learner could also adapt regularly to a patient's changing responses throughout a live operation and facilitate the task of anesthesiologists by prompting them with recommended actions.