Reinforcement Learning Agent Trained as FES Controller in Modified Xia-Law Simulation Robustly Controls Grip Force.

The long-term goal of this research is to improve the adoption of functional electrical stimulation (FES) therapy in rehabilitation clinics. FES has been demonstrated to be an effective tool for rehabilitating stroke and spinal cord injury patients. However, it is not frequently used in clinics, mainly due to long setup times and unfamiliarity with choosing stimulation parameters to get acceptable muscle activation. Here, we develop an FES controller that performs across a range of stimulation parameters without a loss in performance. We trained a reinforcement learning agent with a novel FES-muscle activation model to control contraction by modulating the stimulation amplitude such that control is unaffected by stimulation parameter choice. The trained FES agent was then used to control the grip force of healthy participants using a range of pulse widths, and the root-mean-square error (RMSE) was used to quantify the FES agent's accuracy with each pulse width. The FES agent was tested using 200 μs, 300 μs, and 400 μs pulse widths and achieved root-mean-square errors of 3.50% of the maximum evoked contraction (MEC), 3.59% MEC, and 3.45 % MEC, respectively. We found that the agent's control of grip force was robust and unaffected by the selected pulse width. These results can help guide the development of FES systems that are not only accurate but also simple to use.Clinical Relevance- This reduces the complexity of FES systems, which will decrease the time and training required to set up FES systems and increase clinical acceptance of FES therapy.