ArticleNature human behaviour2025
A unified acoustic-to-speech-to-language embedding space captures the neural basis of natural language processing in everyday conversations.
Article in Nature human behaviour, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 21 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
21 citing papers in PubMed.
- Linguistics and human brain: a perspective of computational neuroscience.Cognitive neurodynamics · 2026Review
- Review
- Article
- A Fine-grained Spatiotemporal ECoG Dataset during Speech Perception in Tonal Language.Scientific data · 2026Article
- Speech Neurophysiology in Realistic Contexts: Big Hype or Big Leap?The European journal of neuroscience · 2026Review
- Electrospun Surface-Modified Epidermal Strain Sensors Enable Silent Speech and Hand Gesture Recognition for Virtual Reality Interaction.Nanomaterials (Basel, Switzerland) · 2026Article
- Article
- Aligning brains into a shared space improves their alignment with large language models.Nature computational science · 2026Article
- Revealing spatiotemporal neural activation patterns in electrocorticography recordings of human speech production by mutual information.Neuroscience informatics · 2025Article
- Learning with others: teacher-learner brain synchrony depends on mutual gaze and joint attention.Cerebral cortex (New York, N.Y. : 1991) · 2025Article
- Real-time decoding of full-spectrum Chinese using brain-computer interface.Science advances · 2025Article
- Temporal propagation of neural state boundaries in naturalistic context.Cerebral cortex (New York, N.Y. : 1991) · 2025Article
- Conversational content is organized across multiple timescales in the brain.Nature human behaviour · 2025Article
- Linguistic coupling between neural systems for speech production and comprehension during real-time dyadic conversations.bioRxiv : the preprint server for biology · 2025Article
- The "Podcast" ECoG dataset for modeling neural activity during natural language comprehension.Scientific data · 2025Article
- Cortical language areas are coupled via a soft hierarchy of model-based linguistic features.bioRxiv : the preprint server for biology · 2025Article
- Neural encoding of real world face perception.ArXiv · 2025Article
- Uncovering a Timescale Hierarchy by Studying the Brain in a Natural Context.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2025Article
- The "Podcast" ECoG dataset for modeling neural activity during natural language comprehension.bioRxiv : the preprint server for biology · 2025Article
- BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activity.Advances in neural information processing systems · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
23 authors.
Funding
Abstract
This study introduces a unified computational framework connecting acoustic, speech and word-level linguistic structures to study the neural basis of everyday conversations in the human brain. We used electrocorticography to record neural signals across 100 h of speech production and comprehension as participants engaged in open-ended real-life conversations. We extracted low-level acoustic, mid-level speech and contextual word embeddings from a multimodal speech-to-text model (Whisper). We developed encoding models that linearly map these embeddings onto brain activity during speech production and comprehension. Remarkably, this model accurately predicts neural activity at each level of the language processing hierarchy across hours of new conversations not used in training the model. The internal processing hierarchy in the model is aligned with the cortical hierarchy for speech and language processing, where sensory and motor regions better align with the model's speech embeddings, and higher-level language areas better align with the model's language embeddings. The Whisper model captures the temporal sequence of language-to-speech encoding before word articulation (speech production) and speech-to-language encoding post articulation (speech comprehension). The embeddings learned by this model outperform symbolic models in capturing neural activity supporting natural speech and language. These findings support a paradigm shift towards unified computational models that capture the entire processing hierarchy for speech comprehension and production in real-world conversations.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.