Familiality of behavioral flexibility and response inhibition deficits in autism spectrum disorder (ASD).

Background: Diminished cognitive control, including reduced behavioral flexibility and behavioral response inhibition, has been repeatedly documented in autism spectrum disorder (ASD). We evaluated behavioral flexibility and response inhibition in probands and their parents using a family trio design to determine the extent to which these cognitive control impairments represent familial traits associated with ASD. Methods: We examined 66 individuals with ASD (probands), 135 unaffected biological parents, and 76 typically developing controls. Participants completed a probabilistic reversal learning task (PRL) and a stop-signal task (SST) to assess behavioral flexibility and response inhibition respectively. Rates of PRL and SST errors were examined across groups, within families, and in relation to clinical and subclinical traits of ASD. Based on prior findings that subclinical broader autism phenotypic (BAP) traits may co-segregate within families and reflect heritable risk factors, we also examined whether cognitive control deficits were more prominent in families in which parents showed BAP features (BAP+). Results: Probands and parents each showed increased rates of PRL and SST errors relative to controls. Error rates across tasks were not related. SST error rates inter-correlated among probands and their parents. PRL errors were more severe in BAP+ parents and their children relative to BAP- parents and their children. For probands of BAP+ parents, PRL and SST error rates were associated with more severe social-communication abnormalities and repetitive behaviors, respectively. Conclusion: Reduced behavioral flexibility and response inhibition are present among probands and their unaffected parents, but represent unique familial deficits associated with ASD that track with separate clinical issues. Specifically, behavioral response inhibition impairments are familial in ASD and manifest independently from parental subclinical features. In contrast, behavioral flexibility deficits are selectively present in families with BAP characteristics, suggesting they co-segregate in families with parental subclinical social, communication, and rigid personality traits. Together, these findings provide evidence that behavioral flexibility and response inhibition impairments track differentially with ASD risk mechanisms and related behavioral traits.