ArticlePLoS computational biology2023
Emergence of belief-like representations through reinforcement learning.
Article in PLoS computational biology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed.
- Decoding behavior with minimal and interpretable agent models.PLoS computational biology · 2026Article
- Mesocorticostriatal Reinforcement Learning of State Representation and Value with Implications for the Mechanisms of Schizophrenia.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2026Article
- Dopamine Dynamics in the Nucleus Accumbens Reflect Confidence in Detecting the Occurrence and Nonoccurrence of Visual Signals in Perceptual Decision-Making.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2026Article
- Data-derived agents reveal dynamical reservoirs in mouse cortex for adaptive behavior.bioRxiv : the preprint server for biology · 2026Article
- Contextual inference through flexible integration of environmental features and behavioural outcomes.PLoS computational biology · 2026Article
- Memory erasure by dopamine-gated retrospective learning.bioRxiv : the preprint server for biology · 2026Article
- Homeostasis after injury: How intertwined inference and control underpin post-injury pain and behaviour.PLoS computational biology · 2026Article
- Neural mechanisms of flexible perceptual inference.PLoS computational biology · 2025Article
- Review
- Article
- Individual differences in decision-making shape how mesolimbic dopamine regulates choice confidence and change-of-mind.Nature neuroscience · 2025Article
- Striatal Gradient in Value-Decay Explains Regional Differences in Dopamine Patterns and Reinforcement Learning Computations.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2025Article
- Nucleus accumbens dopamine release reflects Bayesian inference during instrumental learning.PLoS computational biology · 2025Article
- What dopamine teaches depends on what the brain believes.Nature neuroscience · 2025Article
- Prospective contingency explains behavior and dopamine signals during associative learning.Nature neuroscience · 2025Article
- The devilish details affecting TDRL models in dopamine research.Trends in cognitive sciences · 2025Review
- A corticostriatal learning mechanism linking excess striatal dopamine and auditory hallucinations.bioRxiv : the preprint server for biology · 2025Article
- Expectancy-related changes in firing of dopamine neurons depend on hippocampus.Nature communications · 2024Article
- Inductive biases of neural network modularity in spatial navigation.Science advances · 2024Article
- Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time.Nature communications · 2024Article
Corrections and comments
- Update of
Authors and funding
6 authors.
Funding
Abstract
To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using only incomplete state information. Previous work suggests that animals estimate value in partially observable tasks by first forming "beliefs"-optimal Bayesian estimates of the hidden states in the task. Although this is one way to solve the problem of partial observability, it is not the only way, nor is it the most computationally scalable solution in complex, real-world environments. Here we show that a recurrent neural network (RNN) can learn to estimate value directly from observations, generating reward prediction errors that resemble those observed experimentally, without any explicit objective of estimating beliefs. We integrate statistical, functional, and dynamical systems perspectives on beliefs to show that the RNN's learned representation encodes belief information, but only when the RNN's capacity is sufficiently large. These results illustrate how animals can estimate value in tasks without explicitly estimating beliefs, yielding a representation useful for systems with limited capacity.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.