ArticleAdvances in methods and practices in psychological science
Putting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction.
Article in Advances in methods and practices in psychological science. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Voice Patterns as Markers of Schizophrenia: Building a Cumulative Generalizable Approach Via a Cross-Linguistic and Meta-analysis Based Investigation.Schizophrenia bulletin · 2023Pooled it
- Human Values Across the Lifespan: Age-Graded Differences at Three Hierarchical Levels and What We Can Learn From Them.Personality & social psychology bulletin · 2026Article
- Vocal Markers of Schizophrenia: Assessing the Generalizability of Machine Learning Models and Their Clinical Applicability.Schizophrenia bulletin · 2026Article
- Cognitive Strategy Accounts for Failure on a Relational Memory Task in Non-Human Primates.Hippocampus · 2026Article
- Article
- Personal and institutional factors as determinants of civic engagement among university students.Frontiers in psychology · 2025Article
- Machine learning identifies key individual and nation-level factors predicting climate-relevant beliefs and behaviors.npj climate action · 2025Article
- Predicting Mood Based on the Social Context Measured Through the Experience Sampling Method, Digital Phenotyping, and Social Networks.Administration and policy in mental health · 2024Article
- Never miss a beep: Using mobile sensing to investigate (non-)compliance in experience sampling studies.Behavior research methods · 2024Article
- Using machine learning to predict judgments on Western visual art along content-representational and formal-perceptual attributes.PloS one · 2024Article
- A data-driven approach shows that individuals' characteristics are more important than their networks in predicting fertility preferences.Royal Society open science · 2023Article
- Leakage and the reproducibility crisis in machine-learning-based science.Patterns (New York, N.Y.) · 2023Article
- Are Small Effects the Indispensable Foundation for a Cumulative Psychological Science? A Reply to Götz et al. (2022).Perspectives on psychological science : a journal of the Association for Psychological Science · 2023Article
- Testing, explaining, and exploring models of facial expressions of emotions.Science advances · 2023Article
- Predicting How Well Adolescents Get Along with Peers and Teachers: A Machine Learning Approach.Journal of youth and adolescence · 2022Article
- The Dual Mechanisms of Cognitive Control dataset, a theoretically-guided within-subject task fMRI battery.Scientific data · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
Abstract
Consensus on standards for evaluating models and theories is an integral part of every science. Nonetheless, in psychology, relatively little focus has been placed on defining reliable communal metrics to assess model performance. Evaluation practices are often idiosyncratic and are affected by a number of shortcomings (e.g., failure to assess models' ability to generalize to unseen data) that make it difficult to discriminate between good and bad models. Drawing inspiration from fields such as machine learning and statistical genetics, we argue in favor of introducing common benchmarks as a means of overcoming the lack of reliable model evaluation criteria currently observed in psychology. We discuss a number of principles benchmarks should satisfy to achieve maximal utility, identify concrete steps the community could take to promote the development of such benchmarks, and address a number of potential pitfalls and concerns that may arise in the course of implementation. We argue that reaching consensus on common evaluation benchmarks will foster cumulative progress in psychology and encourage researchers to place heavier emphasis on the practical utility of scientific models.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.