ArticlePhilosophical transactions of the Royal Society of London. Series B, Biological sciences2020
Linguistic generalization and compositionality in modern artificial neural networks.
Article in Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Neuro-symbolic NLP: taxonomy, assessment, and directions.Frontiers in artificial intelligence · 2026Pooled it
- Universal Latent Representation in Finite Ring Continuum.Entropy (Basel, Switzerland) · 2025Article
- Large language models can segment narrative events similarly to humans.Behavior research methods · 2025Article
- Making sense of transformer success.Frontiers in artificial intelligence · 2025Article
- Deep neural networks and humans both benefit from compositional language structure.Nature communications · 2024Article
- Flexible multitask computation in recurrent networks utilizes shared dynamical motifs.Nature neuroscience · 2024Article
- Shared functional specialization in transformer-based language models and the human brain.Nature communications · 2024Article
- How deep is the brain? The shallow brain hypothesis.Nature reviews. Neuroscience · 2023Review
- Evidence of a predictive coding hierarchy in the human brain listening to speech.Nature human behaviour · 2023Article
- Compositional Sequence Generation in the Entorhinal-Hippocampal System.Entropy (Basel, Switzerland) · 2022Article
- Deep language algorithms predict semantic comprehension from brain activity.Scientific reports · 2022Article
- A Computational Neural Network Model for College English Grammar Correction.Computational intelligence and neuroscience · 2022Article
- Can Recurrent Neural Networks Validate Usage-Based Theories of Grammar Acquisition?Frontiers in psychology · 2022Review
- Schrödinger's tree-On syntax and neural language models.Frontiers in artificial intelligence · 2022Review
- The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension.Scientific data · 2021Article
- Disentangling Semantic Composition and Semantic Association in the Left Temporal Lobe.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2021Article
- Theory Before the Test: How to Build High-Verisimilitude Explanatory Theories in Psychological Science.Perspectives on psychological science : a journal of the Association for Psychological Science · 2021Article
- Grammatical category and the neural processing of phrases.Scientific reports · 2021Article
- Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks.Neuron · 2020Review
- Modelling meaning composition from formalism to mechanism.Philosophical transactions of the Royal Society of London. Series B, Biological sciences · 2020Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
No grant is acknowledged in the PubMed record.
Abstract
In the last decade, deep artificial neural networks have achieved astounding performance in many natural language-processing tasks. Given the high productivity of language, these models must possess effective generalization abilities. It is widely assumed that humans handle linguistic productivity by means of algebraic compositional rules: are deep networks similarly compositional? After reviewing the main innovations characterizing current deep language-processing networks, I discuss a set of studies suggesting that deep networks are capable of subtle grammar-dependent generalizations, but also that they do not rely on systematic compositional rules. I argue that the intriguing behaviour of these devices (still awaiting a full understanding) should be of interest to linguists and cognitive scientists, as it offers a new perspective on possible computational strategies to deal with linguistic productivity beyond rule-based compositionality, and it might lead to new insights into the less systematic generalization patterns that also appear in natural language. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.