ArticleRoyal Society open science2022
Citation counts and journal impact factors do not capture some indicators of research quality in the behavioural and brain sciences.
Article in Royal Society open science, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 32 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
32 citing papers in PubMed.
- Orthopaedic publications receive fewer citations than general medicine articles: a scientometric analysis of temporal trends.Journal of orthopaedics and traumatology : official journal of the Italian Society of Orthopaedics and Traumatology · 2026Review
- Prevalence of hyperprolific authors in sports medicine and musculoskeletal health and implications on research attention.PloS one · 2026Article
- Scientific Impact and Its Role in Scientific Reasoning.Journal of Intelligence · 2025Article
- Ending publication bias: A values-based approach to surface null and negative results.PLoS biology · 2025Article
- Global trends and research landscape of arthroscopic therapy and exercise rehabilitation in meniscal tear: a bibliometric analysis.Journal of orthopaedic surgery and research · 2025Article
- Journals Operating Predatory Practices Are Systematically Eroding the Science Ethos: A Gate and Code Strategy to Minimise Their Operating Space and Restore Research Best Practice.Microbial biotechnology · 2025Article
- Normothermic machine perfusion in liver transplantation: a bibliometric analysis of the top 100 most cited articles.Annals of medicine and surgery (2012) · 2025Review
- Correlation between Altmetric Attention Scores and citation scores across the high impact-factor journals each in Medicine, Surgery, and Anaesthesia.British journal of anaesthesia · 2025Article
- The misalignment of incentives in academic publishing and implications for journal reform.Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- Article
- Promotion and tenure for medical physicists should be based on article specific measures and not on journal impact factor.Journal of applied clinical medical physics · 2024Article
- Librarian involvement on knowledge synthesis articles and its relationship to article citation count and Journal Impact Factor.The journal of the Canadian Health Libraries Association · 2024Article
- The artificial intelligence revolution...in unethical publishing: Will AI worsen our dysfunctional publishing system?The Journal of general physiology · 2024Article
- Exploring the connection between EU-funded research and methodological approaches: insights from a retrospective analysis.Journal of translational medicine · 2024Article
- Improving the quality of publications in and advancing the paradigms of clinical and social pharmacy practice research: the Granada Statements.European journal of hospital pharmacy : science and practice · 2024Article
- Evaluation and Comparison of Academic Impact and Disruptive Innovation Level of Medical Journals: Bibliometric Analysis and Disruptive Evaluation.Journal of medical Internet research · 2024Article
- A memory-theoretic account of citation propagation.Royal Society open science · 2024Article
- Towards responsible research assessment: How to reward research quality.PLoS biology · 2024Article
- Article
- An analysis of the effects of sharing research data, code, and preprints on citations.PloS one · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Citation data and journal impact factors are important components of faculty dossiers and figure prominently in both promotion decisions and assessments of a researcher's broader societal impact. Although these metrics play a large role in high-stakes decisions, the evidence is mixed about whether they are strongly correlated with indicators of research quality. We use data from a large-scale dataset comprising 45 144 journal articles with 667 208 statistical tests and data from 190 replication attempts to assess whether citation counts and impact factors predict three indicators of research quality: (i) the accuracy of statistical reporting, (ii) the evidential value of the reported data and (iii) the replicability of a given experimental result. Both citation counts and impact factors were weak and inconsistent predictors of research quality, so defined, and sometimes negatively related to quality. Our findings raise the possibility that citation data and impact factors may be of limited utility in evaluating scientists and their research. We discuss the implications of these findings in light of current incentive structures and discuss alternative approaches to evaluating research.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.