The Serotonin 1B Receptor Modulates Striatal Activity Differentially Based on Behavioral Context.

The dorsomedial striatum (DMS) is critical for both motivating and inhibiting behavioral responses. The region integrates inputs from the cortex, thalamus, and other subcortical structures including midbrain dopamine neurons. Though less studied, serotonin neurons from the dorsal raphe nucleus also richly innervate the DMS, which expresses nearly all 14 serotonin receptor subtypes. Slice electrophysiology shows that the serotonin 1B receptor (5-HT1BR) impacts DMS physiology and plasticity, and behavioral experiments show that 5-HT1BR expression modulates impulsivity and other DMS-dependent reward-related behaviors. In these studies, our goal was to investigate the effects of 5-HT1BR on the DMS in vivo. Using a genetic 5-HT1BR loss-of-function mouse model, we examined calcium activity of individual medium spiny neurons (MSNs) in the DMS of both males and females during operant tasks focusing on responses to actions, reward, and waiting. We found that knock-out of 5-HT1BRs resulted in different effects on MSN calcium activity depending on behavioral state. Specifically, mice lacking 5-HT1BRs showed significantly more inhibition of MSN calcium activity during the rewards, but more cells with excitatory calcium responses during the delay period of the trial. This suggests that serotonin, acting via 5-HT1BRs, may recruit MSN activity in response to reward but inhibit MSN activity during waiting. These results highlight the importance of in vivo studies for understanding the functional role of DMS serotonin in reward-related behavior. Overall our results demonstrate that serotonin can modulate the DMS in a behavioral state-specific manner, potentially providing a mechanism for how serotonin effects on behavior are context dependent.