Evidence map›Paper›PMID 33400613›Full record

ArticlePerspectives on psychological science : a journal of the Association for Psychological Science2021

Heterogeneity of Research Results: A New Perspective From Which to Assess and Promote Progress in Psychological Science.

Audrey Helen Linden, Johannes Hönekopp

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Perspectives on psychological science : a journal of the Association for Psychological Science, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 53 papers, 15 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
53citing papers in PubMed, 15 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

53 citing papers in PubMed, 15 syntheses or guidelines pooled it.

  1. Outcomes of Left Bundle Branch Area Pacing in Heart Failure Patients: A Systematic Review.Annals of noninvasive electrocardiology : the official journal of the International Society for Holter and Noninvasive Electrocardiology, Inc · 2026
    Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Article
  17. Review
  18. Positive affect as a developmental mediator of early adversity and internalizing psychopathology.Journal of child psychology and psychiatry, and allied disciplines · 2026
    Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

2 authors.

Audrey Helen LindenDepartment of Psychology, Northumbria University.
Johannes HönekoppDepartment of Psychology, Northumbria University.ORCID 0000-0002-0613-6801

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Heterogeneity emerges when multiple close or conceptual replications on the same subject produce results that vary more than expected from the sampling error. Here we argue that unexplained heterogeneity reflects a lack of coherence between the concepts applied and data observed and therefore a lack of understanding of the subject matter. Typical levels of heterogeneity thus offer a useful but neglected perspective on the levels of understanding achieved in psychological science. Focusing on continuous outcome variables, we surveyed heterogeneity in 150 meta-analyses from cognitive, organizational, and social psychology and 57 multiple close replications. Heterogeneity proved to be very high in meta-analyses, with powerful moderators being conspicuously absent. Population effects in the average meta-analysis vary from small to very large for reasons that are typically not understood. In contrast, heterogeneity was moderate in close replications. A newly identified relationship between heterogeneity and effect size allowed us to make predictions about expected heterogeneity levels. We discuss important implications for the formulation and evaluation of theories in psychology. On the basis of insights from the history and philosophy of science, we argue that the reduction of heterogeneity is important for progress in psychology and its practical applications, and we suggest changes to our collective research practice toward this end.

Indexed as

Behavioral ResearchPsychologyFemaleHumansMaleMeta-Analysis as Topicheterogeneitymeta-analysisphilosophy of sciencepsychological researchreplicationstatistical power

Identifiers

PMID33400613
PMCPMC7961629

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.