ArticleNature human behaviour2024
Testing theory of mind in large language models and humans.
Article in Nature human behaviour, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 58 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
58 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Human-like conversational agents as social partners: a scoping review of socioaffective mechanisms, well-being outcomes, risks and governance in the post-Turing era.Frontiers in artificial intelligence · 2026Pooled it
- Algorithmic Reflectivity in Large Language Models: Methodological Study of Linguistic Structure in Mentalization.JMIR AI · 2026Article
- Developing and Evaluating the Use of ChatGPT as a Screening Tool for Nurses Conducting Structured Literature Reviews: Proof of Concept Study Results.Journal of clinical nursing · 2026Article
- Disentangling interaction and bias effects in opinion dynamics of large language models.Nature communications · 2026Article
- Frameworks, Methodologies, and Tools for Evaluating Large Language Models in Digital Mental Health Interventions: Protocol for a Scoping Review.JMIR research protocols · 2026Article
- Perceived humanness and empathy are dissociated in relationship advice generated by a large language model.Nature communications · 2026Article
- Large Language Models Persuade Without Planning Theory of Mind.Cognitive science · 2026Article
- AI agents can coordinate via majority-following beyond human scale.Science advances · 2026Article
- Generics in science communication: Misaligned interpretations across laypeople, scientists, and large language models.Public understanding of science (Bristol, England) · 2026Article
- Cortical knowledge structures guide word concept learning.Nature communications · 2026Article
- The AI interviewer: multi-faceted evaluation of adaptive questioning by large language models.Scientific reports · 2026Article
- Collective and augmented intelligence outperform artificial intelligence on emotion recognition tests.Scientific reports · 2026Article
- A Verbal Behavior Analysis of Theory of Mind: Conceptual and Applied Implications.Perspectives on behavior science · 2026Article
- A large-scale evaluation of commonsense knowledge in humans and large language models.PNAS nexus · 2026Article
- A large-scale comparison of divergent creativity in humans and large language models.Nature human behaviour · 2026Article
- How well do large language models mirror human cognition of word concepts?: A comparison of psychological ratings for early-acquired English words.Behavior research methods · 2026Article
- Quantifying uncert-AI-nty: Testing the accuracy of LLMs' confidence judgments.Memory & cognition · 2026Article
- Review
- Influence of solution efficiency and valence of instruction on additive and subtractive solution strategies in humans, GPT-4, and GPT-4o.Communications psychology · 2026Article
- Research on the development of an automated system for psychology questionnaire generation based on large language models.PloS one · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
12 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
At the core of what defines us as humans is the concept of theory of mind: the ability to track other people's mental states. The recent development of large language models (LLMs) such as ChatGPT has led to intense debate about the possibility that these models exhibit behaviour that is indistinguishable from human behaviour in theory of mind tasks. Here we compare human and LLM performance on a comprehensive battery of measurements that aim to measure different theory of mind abilities, from understanding false beliefs to interpreting indirect requests and recognizing irony and faux pas. We tested two families of LLMs (GPT and LLaMA2) repeatedly against these measures and compared their performance with those from a sample of 1,907 human participants. Across the battery of theory of mind tests, we found that GPT-4 models performed at, or even sometimes above, human levels at identifying indirect requests, false beliefs and misdirection, but struggled with detecting faux pas. Faux pas, however, was the only test where LLaMA2 outperformed humans. Follow-up manipulations of the belief likelihood revealed that the superiority of LLaMA2 was illusory, possibly reflecting a bias towards attributing ignorance. By contrast, the poor performance of GPT originated from a hyperconservative approach towards committing to conclusions rather than from a genuine failure of inference. These findings not only demonstrate that LLMs exhibit behaviour that is consistent with the outputs of mentalistic inference in humans but also highlight the importance of systematic testing to ensure a non-superficial comparison between human and artificial intelligences.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.