ArticlePerspectives on psychological science : a journal of the Association for Psychological Science2024
AI Psychometrics: Assessing the Psychological Profiles of Large Language Models Through Psychometric Inventories.
Article in Perspectives on psychological science : a journal of the Association for Psychological Science, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Personality, capability, and context: a 30 year integrative review and theoretical expansion of the Big Five framework.Frontiers in psychology · 2026Pooled it
- Metric Quality of Intelligence Tests in Spain: Evidence from 13 National Test Commission Reviews.Journal of Intelligence · 2026Article
- AI agents can coordinate via majority-following beyond human scale.Science advances · 2026Article
- Simulating insufficient effort responding with large language models: A persona-based prompt engineering approach.Behavior research methods · 2026Article
- Simulated self-assessment in large language models: A psychometric approach to AI self-efficacy.Computers in human behavior. Artificial humans · 2026Article
- Beyond composite scores in chronotype assessment: item-level predictive patterns in the Morningness-Eveningness Questionnaire.Scientific reports · 2026Observational
- On the makeup of high-impact reviews in ecology.Eco-Environment & Health · 2026Article
- Human Shadows in Machine Minds: Quantitative Study Interpreting AI Responses to the Rorschach Test.JMIR mental health · 2026Article
- A multimodal EEG-based psychological risk representation method using spatiotemporal graph attention and adaptive gated fusion.Frontiers in psychiatry · 2026Article
- Improvement of metaphor understanding via a cognitive linguistic model based on hierarchical classification and artificial intelligence SVM.Scientific reports · 2025Article
- Kernels of selfhood: GPT-4o shows humanlike patterns of cognitive dissonance moderated by free choice.Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- AI can outperform humans in predicting correlations between personality items.Communications psychology · 2025Article
- Readiness Evaluation for AI-Mental Health Deployment and Implementation (READI): A Review and Proposed Framework.Technology, mind, and behavior · 2025Article
- How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test.Health data science · 2025Article
- Personality testing of large language models: limited temporal stability, but highlighted prosociality.Royal Society open science · 2024Article
- Can Generative AI improve social science?Proceedings of the National Academy of Sciences of the United States of America · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
We illustrate how standard psychometric inventories originally designed for assessing noncognitive human traits can be repurposed as diagnostic tools to evaluate analogous traits in large language models (LLMs). We start from the assumption that LLMs, inadvertently yet inevitably, acquire psychological traits (metaphorically speaking) from the vast text corpora on which they are trained. Such corpora contain sediments of the personalities, values, beliefs, and biases of the countless human authors of these texts, which LLMs learn through a complex training process. The traits that LLMs acquire in such a way can potentially influence their behavior, that is, their outputs in downstream tasks and applications in which they are employed, which in turn may have real-world consequences for individuals and social groups. By eliciting LLMs' responses to language-based psychometric inventories, we can bring their traits to light. Psychometric profiling enables researchers to study and compare LLMs in terms of noncognitive characteristics, thereby providing a window into the personalities, values, beliefs, and biases these models exhibit (or mimic). We discuss the history of similar ideas and outline possible psychometric approaches for LLMs. We demonstrate one promising approach, zero-shot classification, for several LLMs and psychometric inventories. We conclude by highlighting open challenges and future avenues of research for AI Psychometrics.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.