ArticlePNAS nexus2024
Perils and opportunities in using large language models in psychological research.
Article in PNAS nexus, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 32 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
32 citing papers in PubMed.
- Simulated responses from large language models are weak proxies for human economic preferences across 74 countries.Nature human behaviour · 2026Article
- Large language models as psychometric laboratories? Maybe not yet. Cautionary evidence from synthetic personality data.PCN reports : psychiatry and clinical neurosciences · 2026Article
- Article
- The Internalized Ableism Inventory: Scale development using a hybrid artificial intelligence and community-based participatory research design.Rehabilitation psychology · 2026Article
- Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings.Behavior research methods · 2026Article
- A flexible behavioral method for measuring human and artificial intelligence alignment using representational similarity analysis.iScience · 2026Article
- Using natural language processing to track negative emotions in the daily lives of adolescents.Journal of psychopathology and clinical science · 2026Article
- Reframing the Performance and Ethics of Empathic AI: Wisdom of the Crowd and Placebos.Perspectives on psychological science : a journal of the Association for Psychological Science · 2026Article
- Can AI help reduce prejudice? Evaluating the effectiveness of AI-powered personalized persuasion on support for transgender rights.PNAS nexus · 2026Article
- Multi-Agent LLMs for Occupational Profiling: Psychometric Validation on 1636 Chinese Occupations.Behavioral sciences (Basel, Switzerland) · 2026Article
- The application of intelligent generation model for international discourse of grand canal culture based on artificial intelligence and BPNN model.Scientific reports · 2026Article
- Youth supporting youth online: A content analysis of peer counselling at the online chat of @ease.Internet interventions · 2026Article
- Are We Helping Workers Reskill for the Future of Work? Using AI to Explore the Alignment of Online Course Offerings and Job Skill Requirements.Journal of Intelligence · 2026Article
- Moral stereotyping in large language models.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Profiles and Temporal Dynamics of Personality-Related Language in a 16-year Professional Email Archive: A Subject-Specific Study (2008-2024).Journal for person-oriented research · 2026Article
- ChatGPT does not replicate human moral judgments: the importance of examining metrics beyond correlation to assess agreement.Scientific reports · 2025Article
- Estimating the effect size of moral contagion in online networks: A pre-registered replication and meta-analysis.PNAS nexus · 2025Article
- A Prompt Engineering Method for Generating Emotional Images for Psychological Research.Affective science · 2025Article
- Evaluation of Six Large Language Models for Clinical Decision Support: Application in Transfusion Decision-making for RhD Blood-type Patients.Annals of laboratory medicine · 2025Article
- New opportunities and challenges for conservation evidence synthesis from advances in natural language processing.Conservation biology : the journal of the Society for Conservation Biology · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
The emergence of large language models (LLMs) has sparked considerable interest in their potential application in psychological research, mainly as a model of the human psyche or as a general text-analysis tool. However, the trend of using LLMs without sufficient attention to their limitations and risks, which we rhetorically refer to as "GPTology", can be detrimental given the easy access to models such as ChatGPT. Beyond existing general guidelines, we investigate the current limitations, ethical implications, and potential of LLMs specifically for psychological research, and show their concrete impact in various empirical studies. Our results highlight the importance of recognizing global psychological diversity, cautioning against treating LLMs (especially in zero-shot settings) as universal solutions for text analysis, and developing transparent, open methods to address LLMs' opaque nature for reliable, reproducible, and robust inference from AI-generated data. Acknowledging LLMs' utility for task automation, such as text annotation, or to expand our understanding of human psychology, we argue for diversifying human samples and expanding psychology's methodological toolbox to promote an inclusive, generalizable science, countering homogenization, and over-reliance on LLMs.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.