Effort and Substance Use: Differentiating Tobacco Use Through Reinforcement Learning of Effort Based Decision Making.

backgroundEffort-based decision making evaluates rewards relative to the effort required to obtain it, an important process of healthy goal-directed motivation and behavior. Computational models provide mechanistic insights underlying choice behavior and potential alterations in neuropsychiatric disorders, including substance use disorders. We applied computational models to effort-based choice behavior to characterize underlying decision processes and if these mechanisms differ by substance use status.

methodsParticipants completed the Effort Expenditure for Rewards Task, choosing between low- and high-effort options for monetary rewards varying in magnitude and probability. Participants met criteria for no tobacco use (n = 23), current tobacco use disorder (n = 26), former tobacco use disorder (n = 22), and tobacco and opioid use disorder (n = 29). Computational models from two families, Subjective Value and Reinforcement Learning, were fit and compared. Parameters from the best-fitting model underwent principal components analysis and linear discriminant analysis.

resultsTemporal difference reinforcement learning model demonstrated greater model evidence and predictive accuracy, indicating better fit to effort-based choice behavior. Principal components analysis revealed meaningful multivariate distinctions: PC1 differentiated all groups except individuals without tobacco use versus individuals with tobacco use disorder; PC3 distinguished tobacco and opioid use disorder from all other groups. Linear discriminant analysis demonstrated group separation with 76% classification accuracy.

conclusionsA reinforcement learning framework better explained participants' effort-based choice behavior. Substance use status relates to dynamic behavioral changes (i.e. learning) as measured by the multivariate combination of learning rate, future discounting, and choice temperature.