ArticlePLoS computational biology2015
Theory of choice in bandit, information sampling and foraging tasks.
Article in PLoS computational biology, 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 62 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
62 citing papers in PubMed, 1 synthesis or guideline pooled it.
- A review on exploration-exploitation trade-off in psychiatric disorders.BMC psychiatry · 2025Pooled it
- Novelty-seeking and directed exploration scale with dissociable social network outcomes and brain systems in older adults.bioRxiv : the preprint server for biology · 2026Article
- Spatiotemporal Decoding of Explore-Exploit Decisions in the Human Brain.bioRxiv : the preprint server for biology · 2026Article
- AI-Discovered Cognitive Models Reveal Novel Insights into Human and Animal Learning.bioRxiv : the preprint server for biology · 2026Article
- Stochastic choice drives variability in patch foraging decisions in humans and rats.Communications psychology · 2026Article
- A bio-inspired minimal model for non-stationary K-armed bandits.Biological cybernetics · 2026Article
- Rats pursue food and leisure following the same rational principles.Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- Evolving choice hysteresis in reinforcement learning: Comparing the adaptive value of positivity bias and gradual perseveration.Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- Individual differences in sequential decision-making.bioRxiv : the preprint server for biology · 2025Article
- Sophisticated Learning: A novel algorithm for active learning during model-based planning.ArXiv · 2025Article
- Success-efficient/failure-safe strategy for hierarchical reinforcement motor learning.PLoS computational biology · 2025Article
- Article
- Meta-Analysis Reveals That Explore-Exploit Decisions are Dissociable by Activation in the Dorsal Lateral Prefrontal Cortex, Anterior Insula, and the Dorsal Anterior Cingulate Cortex.bioRxiv : the preprint server for biology · 2025Article
- Fluctuations in Sequential Many-Alternative Decisions Reveal Strategies Beyond Immediate Reward Maximisation.Journal of cognition · 2025Article
- Biased expectations about future choice options predict sequential economic decisions.Communications psychology · 2024Article
- A causal role of the right dorsolateral prefrontal cortex in random exploration.Scientific reports · 2024Article
- Complex behavior from intrinsic motivation to occupy future action-state path space.Nature communications · 2024Article
- Variability and harshness shape flexible strategy-use in support of the constrained flexibility framework.Scientific reports · 2024Article
- Motor System-Dependent Effects of Amygdala and Ventral Striatum Lesions on Explore-Exploit Behaviors.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2024Article
- Childhood unpredictability and the development of exploration.Proceedings of the National Academy of Sciences of the United States of America · 2023Article
2 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
Abstract
Decision making has been studied with a wide array of tasks. Here we examine the theoretical structure of bandit, information sampling and foraging tasks. These tasks move beyond tasks where the choice in the current trial does not affect future expected rewards. We have modeled these tasks using Markov decision processes (MDPs). MDPs provide a general framework for modeling tasks in which decisions affect the information on which future choices will be made. Under the assumption that agents are maximizing expected rewards, MDPs provide normative solutions. We find that all three classes of tasks pose choices among actions which trade-off immediate and future expected rewards. The tasks drive these trade-offs in unique ways, however. For bandit and information sampling tasks, increasing uncertainty or the time horizon shifts value to actions that pay-off in the future. Correspondingly, decreasing uncertainty increases the relative value of actions that pay-off immediately. For foraging tasks the time-horizon plays the dominant role, as choices do not affect future uncertainty in these tasks.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.