Evidence map›Paper›PMID 24411645›Full record

ArticleLancet (London, England)2014

Increasing value and reducing waste in research design, conduct, and analysis.

John P A Ioannidis, Sander Greenland, Mark A Hlatky, Muin J Khoury, Malcolm R Macleod, David Moher, Kenneth F Schulz, Robert Tibshirani

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Lancet (London, England), 2014. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 638 papers, 26 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
638citing papers in PubMed, 26 pooled it
137.8field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

638 citing papers in PubMed, 26 syntheses or guidelines pooled it, 1,629 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Colchicine for the primary prevention of cardiovascular events.The Cochrane database of systematic reviews · 2025
    Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

578 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

8 authors at 5 institutions in 3 countries.

John P A IoannidisStanford Prevention Research Center, Department of Medicine, School of Medicine, Stanford University, Stanford, CA, USA; Division of Epidemiology, School of Medicine, Stanford University, Stanford, CA, USA; Department of Statistics, School of Humanities and Sciences, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA. Electronic address: jioannid@stanford.edu.
Sander GreenlandDepartment of Epidemiology and Department of Statistics, UCLA School of Public Health, Los Angeles, CA, USA.
Mark A HlatkyDivision of Cardiovascular Medicine, Department of Medicine, School of Medicine, Stanford University, Stanford, CA, USA; Division of Health Services Research, Stanford University, Stanford, CA, USA.
Muin J KhouryOffice of Public Health Genomics, Centers for Disease Control and Prevention, Atlanta, GA, USA; Epidemiology and Genomics Research Program, National Cancer Institute, Rockville, MD, USA.
Malcolm R MacleodDepartment of Clinical Neurosciences, University of Edinburgh School of Medicine, Edinburgh, UK.
David MoherClinical Epidemiology Program, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, ON, Canada; Department of Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Kenneth F SchulzFHI 360, Durham, NC, USA; Department of Obstetrics and Gynecology, University of North Carolina School of Medicine, Chapel Hill, NC, USA.
Robert TibshiraniDepartment of Health Research and Policy, Stanford University, Stanford, CA, USA; Department of Statistics, School of Humanities and Sciences, Stanford University, Stanford, CA, USA.
Stanford University · USFamily Health International 360 · USNational Cancer Institute · USUniversity of Edinburgh · GBUniversity of Ottawa · CA

Funding

Intramural CDC HHS CC999999National Centre for the Replacement, Refinement and Reduction of Animals in Research NC/L000970/1
6 · The paper itself

Abstract

Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.

Indexed as

Reproducibility of ResultsResearch DesignResearch PersonnelRewardStatistics as Topic

Identifiers

PMID24411645
PMCPMC4697939
OpenAlexW1975130349

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.