ArticleNature machine intelligence2025
High-level visual representations in the human brain are aligned with large language models.
Article in Nature machine intelligence, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 26 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
26 citing papers in PubMed.
- Linguistics and human brain: a perspective of computational neuroscience.Cognitive neurodynamics · 2026Review
- Systematic image perturbations reveal persistent gaps between human and machine vision.iScience · 2026Article
- A solution to generalized learning from small training sets found in infants' repeated visual experiences of individual objects.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Multidimensional Feature Tuning in Category Selective Areas of Human Visual Cortex.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2026Article
- Integrating neuroscience across species and scales.Nature neuroscience · 2026Article
- Computational mechanisms transforming visual codes into sparse representations.Neuroscience and biobehavioral reviews · 2026Review
- Triple-N dataset: large-scale fMRI-guided dense recordings of nonhuman primate neural responses to natural scenes.Nature neuroscience · 2026Article
- A 5.0 T Ultra-High-Field fMRI Dataset for Naturalistic Visual Scene Processing.Scientific data · 2026Article
- Conceptual priorities shape individual gaze patterns during naturalistic visual attention.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Fixation duration on natural scenes is explained by memory encoding not processing demand.Nature neuroscience · 2026Article
- Shared representations in brains and models reveal a two-route cortical organization during scene perception.Communications biology · 2026Article
- Representations in vision and language converge in a shared, multidimensional space of perceived similarities.Journal of vision · 2026Article
- Human-like cognitive generalization for large models via mental representation-guided supervision.Nature communications · 2026Article
- Three-dimensional scene boundary representations for wall orientation and distance are represented distinctly in the human visual cortex.PLoS biology · 2026Article
- Combined evidence from artificial neural networks and human brain-lesion models reveals that language modulates vision in human perception.Nature human behaviour · 2026Article
- Narrative context shifts gaze from visual to semantic salience.Communications psychology · 2026Article
- A 7T fMRI dataset of synthetic images for out-of-distribution modeling of vision.Nature communications · 2026Article
- Characterization of the spatiotemporal representations of visual, semantic, and memorability features in the human brain.PLoS biology · 2026Article
- Eye movements during free viewing to maximize scene understanding.Nature communications · 2025Article
- Brain-AI Alignment in Naturalistic Movies.bioRxiv : the preprint server for biology · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
The human brain extracts complex information from visual inputs, including objects, their spatial and semantic interrelations, and their interactions with the environment. However, a quantitative approach for studying this information remains elusive. Here we test whether the contextual information encoded in large language models (LLMs) is beneficial for modelling the complex visual information extracted by the brain from natural scenes. We show that LLM embeddings of scene captions successfully characterize brain activity evoked by viewing the natural scenes. This mapping captures selectivities of different brain areas and is sufficiently robust that accurate scene captions can be reconstructed from brain activity. Using carefully controlled model comparisons, we then proceed to show that the accuracy with which LLM representations match brain representations derives from the ability of LLMs to integrate complex information contained in scene captions beyond that conveyed by individual words. Finally, we train deep neural network models to transform image inputs into LLM representations. Remarkably, these networks learn representations that are better aligned with brain representations than a large number of state-of-the-art alternative models, despite being trained on orders-of-magnitude less data. Overall, our results suggest that LLM embeddings of scene captions provide a representational format that accounts for complex information extracted by the brain from visual inputs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.