Evidence map›Paper›PMID 34077797›Full record

ArticleJournal of clinical epidemiology2021

Methodology over metrics: current scientific standards are a disservice to patients and society.

Ben Van Calster, Laure Wynants, Richard D Riley, Maarten van Smeden, Gary S Collins

Abstract read
In one paragraph

Article in Journal of clinical epidemiology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 46 papers, 4 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
46citing papers in PubMed, 4 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

46 citing papers in PubMed, 4 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Article
  6. Article
  7. Review
  8. Review
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Review
  15. Article
  16. Retractions in Otolaryngology Publications.JAMA otolaryngology-- head & neck surgery · 2025
    Article
  17. Article
  18. Review
  19. Article
  20. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

5 authors.

Ben Van CalsterDepartment of Development and Regeneration, KU Leuven, Leuven, Belgium; Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands; EPI-Centre, KU Leuven, Leuven, Belgium. Electronic address: ben.vancalster@kuleuven.be.
Laure WynantsDepartment of Development and Regeneration, KU Leuven, Leuven, Belgium; EPI-Centre, KU Leuven, Leuven, Belgium; Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, The Netherlands.
Richard D RileyCentre for Prognosis Research, School of Medicine, Keele University, Keele, UK.
Maarten van SmedenJulius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands.
Gary S CollinsCentre for Statistics in Medicine, Nuffield Department of Orthopaedics, Musculoskeletal Sciences, University of Oxford, Oxford, UK; NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, UK; UK EQUATOR Centre, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK.

Funding

Cancer Research UK 27294
6 · The paper itself

Abstract

Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest that most problems stem from an underlying paradox: although methodology is undeniably the backbone of high-quality and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, improved methodological review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.

Indexed as

Biomedical ResearchCOVID-19HumansResearch DesignMethodologyReportingResearch quality

Identifiers

PMID34077797
PMCPMC8795888

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.