Learning optimal treatment strategies for intraoperative hypotension using deep reinforcement learning.

Importance: Traditional methods of surgical decision making heavily rely on human experience and prompt actions, which are variable. A data-driven system that generates treatment recommendations based on patient states can be a substantial asset in perioperative decision-making, as in cases of intraoperative hypotension, for which suboptimal management is associated with acute kidney injury (AKI), a common and morbid postoperative complication. Objective: To develop a Reinforcement Learning (RL) model to recommend optimum dose of intravenous (IV) fluid and vasopressors during surgery to avoid intraoperative hypotension and postoperative AKI. Design setting participants: We retrospectively analyzed 50,021 surgeries from 42,547 adult patients who underwent major surgery at a quaternary care hospital between June 2014 and September 2020. Of these, 34,186 surgeries were used for model training and internal validation while 15,835 surgeries were reserved for testing. We developed an RL model based on Deep Q-Networks to provide optimal treatment suggestions. Exposures: Demographic and baseline clinical characteristics, intraoperative physiologic time series, and total dose of IV fluid and vasopressors were extracted every 15-minutes during the surgery. Main outcomes: In the RL model, intraoperative hypotension (MAP<65 mmHg) and AKI in the first three days following the surgery were considered. Results: The developed model replicated 69% of physician's decisions for the dosage of vasopressors and proposed higher or lower dosage of vasopressors than received in 10% and 21% of the treatments, respectively. In terms of intravenous fluids, the model's recommendations were within 0.05 ml/kg/15 min of the actual dose in 41% of the cases, with higher or lower doses recommended for 27% and 32% of the treatments, respectively. The RL policy resulted in a higher estimated policy value compared to the physicians' actual treatments, as well as random policies and zero-drug policies. The prevalence of AKI was lowest in the patients who received medication dosages that aligned with our agent model's decisions. Conclusions and Relevance: Our findings suggest that implementation of the model's policy has the potential to reduce postoperative AKI and improve other outcomes driven by intraoperative hypotension.