Does predictability modulate the sentence superiority effect? Perhaps - but not as one might predict!

Written words presented within a sequence of words are identified more accurately when this sequence forms a correct sentence or phrase compared with an ungrammatical re-ordering of the same words. Here we examined if this sentence superiority effect (SSE) is modulated by the predictability of the target word given the sentence context. Target words were at positions 2 and 5 in five-word sequences, and either had high or low cloze probabilities (measured by an independent cloze test and further checked using the word-in-context probabilities obtained with a Large Language Model (LLM)). Given that predictability only made sense when considering grammatically correct sequences, we performed two separate analyses. We found: (1) an effect of grammaticality (i.e., we replicated the SSE), (2) a small effect of predictability on responses to grammatically correct sequences, and (3) no interaction between predictability and grammaticality. The impact of predictability on the SSE was then evaluated by comparing the magnitude of the SSE obtained with more predictable words versus less predictable words, with the SSE in each of these conditions being measured using identification accuracy for the same word at the same position in grammatical and ungrammatical sequences. Results revealed no significant modulation of the SSE by word predictability. We conclude that syntactic and semantic constraints, and not predictability per se (as measured by a cloze test or LLM statistics), contribute to the sentence superiority effect. Crucially, this provides evidence against guessing accounts of the SSE according to which predictability should play a key role.