ArticleProceedings. Biological sciences2021
More than words: word predictability, prosody, gesture and mouth movements in natural language comprehension.
Article in Proceedings. Biological sciences, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 29 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
29 citing papers in PubMed.
- Multimodal integration supports neural processing of grammatical negation.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Understanding and Cultivating Effective Listening: A Dialectical Theory of the Tensions Between Intuition and Behavior.Behavioral sciences (Basel, Switzerland) · 2026Article
- Discourse Context and Co-Speech Gestures Jointly Shape Hierarchical Prediction During the Processing of a Multimodal Narrative.Neurobiology of language (Cambridge, Mass.) · 2026Article
- Social interaction shapes and boosts second language learning: virtual reality can show us how.NPJ science of learning · 2025Review
- Prior expectations guide multisensory integration during face-to-face communication.PLoS computational biology · 2025Article
- Understanding discourse in face-to-face settings: The impact of multimodal cues and listening conditions.Journal of experimental psychology. Learning, memory, and cognition · 2025Article
- Co-Speech Hand Gestures Are Used to Predict Upcoming Meaning.Psychological science · 2025Article
- Cross-modal co-occurrence analysis of nonverbal behavior and content words, vocal emotion and prosody in individuals with subclinical depression.BMC psychology · 2025Article
- The ECOLANG Multimodal Corpus of adult-child and adult-adult Language.Scientific data · 2025Article
- Understanding words in context: A naturalistic EEG study of children's lexical processing.Journal of memory and language · 2024Article
- Mouth and facial informativeness norms for 2276 English words.Behavior research methods · 2024Article
- The language network as a natural kind within the broader landscape of the human brain.Nature reviews. Neuroscience · 2024Review
- The impact of visual information in speech perception for individuals with hearing loss: a mini review.Frontiers in psychology · 2024Review
- Gestures speed up responses to questions.Language, cognition and neuroscience · 2024Article
- The role of multimodal cues in second language comprehension.Scientific reports · 2023Article
- Interactionally Embedded Gestalt Principles of Multimodal Human Communication.Perspectives on psychological science : a journal of the Association for Psychological Science · 2023Review
- Electrophysiological evidence for the enhancement of gesture-speech integration by linguistic predictability during multimodal discourse comprehension.Cognitive, affective & behavioral neuroscience · 2023Article
- The multimodal facilitation effect in human communication.Psychonomic bulletin & review · 2023Article
- Explaining flexible continuous speech comprehension from individual motor rhythms.Proceedings. Biological sciences · 2023Article
- The role of iconic gestures and mouth movements in face-to-face communication.Psychonomic bulletin & review · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
The ecology of human language is face-to-face interaction, comprising cues such as prosody, co-speech gestures and mouth movements. Yet, the multimodal context is usually stripped away in experiments as dominant paradigms focus on linguistic processing only. In two studies we presented video-clips of an actress producing naturalistic passages to participants while recording their electroencephalogram. We quantified multimodal cues (prosody, gestures, mouth movements) and measured their effect on a well-established electroencephalographic marker of processing load in comprehension (N400). We found that brain responses to words were affected by informativeness of co-occurring multimodal cues, indicating that comprehension relies on linguistic and non-linguistic cues. Moreover, they were affected by interactions between the multimodal cues, indicating that the impact of each cue dynamically changes based on the informativeness of other cues. Thus, results show that multimodal cues are integral to comprehension, hence, our theories must move beyond the limited focus on speech and linguistic processing.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.