ArticleProceedings of the National Academy of Sciences of the United States of America2024
A Turing test of whether AI chatbots are behaviorally similar to humans.
Article in Proceedings of the National Academy of Sciences of the United States of America, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 40 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
40 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Human-like conversational agents as social partners: a scoping review of socioaffective mechanisms, well-being outcomes, risks and governance in the post-Turing era.Frontiers in artificial intelligence · 2026Pooled it
- Article
- Article
- Naturalistic Decision-Making Patterns in Cardiometabolic Multimorbidity Self-Care: A Qualitative Study.Annals of family medicine · 2026Article
- A flexible behavioral method for measuring human and artificial intelligence alignment using representational similarity analysis.iScience · 2026Article
- A validity-guided workflow for robust large language model research in psychology.Behavior research methods · 2026Article
- Recognising and mitigating LLM Pollution in online behavioural research.Nature communications · 2026Article
- LLM agents overcome the machine penalty when acting fairly but not when acting selfishly or altruistically.National science review · 2026Article
- Training an AI Chatbot to Manage Health in Underserved Populations: Methodological Approach.JMIR AI · 2026Article
- A large-scale evaluation of commonsense knowledge in humans and large language models.PNAS nexus · 2026Article
- Exploring the potential of large language models in nutrition behavior prediction: evidence from college students.Frontiers in nutrition · 2026Article
- Evidence-based construction and validation of an agent-based model of community integrated care for older adults living with dementia.Frontiers in public health · 2026Article
- Evaluating AI-based comprehensive clinical decision support for sepsis and ARDS: protocol for a Clinician Turing Test.BMJ open · 2025Article
- Coordination of network heterogeneity and individual preferences promotes collective fairness.Patterns (New York, N.Y.) · 2025Article
- Article
- Judgments of learning distinguish humans from large language models in predicting memory.Scientific reports · 2025Article
- Spectral Properties of Complex Distributed Intelligence Systems Coupled with an Environment.Entropy (Basel, Switzerland) · 2025Article
- Using large language models to categorize strategic situations and decipher motivations behind human behaviors.Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- Can Large Language Models Simulate Spoken Human Conversations?Cognitive science · 2025Article
- Testing for completions that simulate altruism in early language models.Nature human behaviour · 2025Article
Corrections and comments
- Commented on by
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
We administer a Turing test to AI chatbots. We examine how chatbots behave in a suite of classic behavioral games that are designed to elicit characteristics such as trust, fairness, risk-aversion, cooperation, etc., as well as how they respond to a traditional Big-5 psychological survey that measures personality traits. ChatGPT-4 exhibits behavioral and personality traits that are statistically indistinguishable from a random human from tens of thousands of human subjects from more than 50 countries. Chatbots also modify their behavior based on previous experience and contexts "as if" they were learning from the interactions and change their behavior in response to different framings of the same strategic situation. Their behaviors are often distinct from average and modal human behaviors, in which case they tend to behave on the more altruistic and cooperative end of the distribution. We estimate that they act as if they are maximizing an average of their own and partner's payoffs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.