ArticleInsights into imaging2020
An analysis of key indicators of reproducibility in radiology.
Article in Insights into imaging, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
11 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Reproducibility of Deep Learning Algorithms Developed for Medical Imaging Analysis: A Systematic Review.Journal of digital imaging · 2023Pooled it
- Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data.BMJ (Clinical research ed.) · 2023Pooled it
- Systematic review of research design and reporting of imaging studies applying convolutional neural networks for radiological cancer diagnosis.European radiology · 2021Pooled it
- Why Radiomics Rarely Reaches the Clinic: Reproducibility, Validation, and Evidence Gap-A Critical Narrative Review.Diagnostics (Basel, Switzerland) · 2026Review
- Nationwide data-driven quality improvement in ultrasound medicine: insights from an indicator-based initiative in China.BMJ open quality · 2025Article
- Advancing Medical Imaging Research Through Standardization: The Path to Rapid Development, Rigorous Validation, and Robust Reproducibility.Investigative radiology · 2025Review
- Sample size calculation for data reliability and diagnostic performance: a go-to review.European radiology experimental · 2024Review
- Generalizability of prostate MRI deep learning: does one size fit all data?European radiology · 2023Article
- How often do cancer researchers make their data and code available and what factors are associated with sharing?BMC medicine · 2022Article
- Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility.European radiology · 2022Article
- Accountability in EUS: Is it possible?Endoscopic ultrasoundArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundGiven the central role of radiology in patient care, it is important that radiological research is grounded in reproducible science. It is unclear whether there is a lack of reproducibility or transparency in radiologic research. PURPOSE: To analyze published radiology literature for the presence or lack of key indicators of reproducibility.
methodsThis cross-sectional retrospective study was performed by conducting a search of the National Library of Medicine (NLM) for publications contained within journals in the field of radiology. Our inclusion criteria were being MEDLINE indexed, written in English, and published from January 1, 2014, to December 31, 2018. We randomly sampled 300 publications for this study. A pilot-tested Google form was used to record information from the publications regarding indicators of reproducibility. Following peer-review, we extracted data from an additional 200 publications in an attempt to reproduce our initial results. The additional 200 publications were selected from the list of initially randomized publications.
resultsOur initial search returned 295,543 records, from which 300 were randomly selected for analysis. Of these 300 records, 294 met inclusion criteria and 6 did not. Among the empirical publications, 5.6% (11/195, [3.0-8.3]) contained a data availability statement, 0.51% (1/195) provided clear documented raw data, 12.0% (23/191, [8.4-15.7]) provided a materials availability statement, 0% provided analysis scripts, 4.1% (8/195, [1.9-6.3]) provided a pre-registration statement, 2.1% (4/195, [0.4-3.7]) provided a protocol statement, and 3.6% (7/195, [1.5-5.7]) were pre-registered. The validation study of the 5 key indicators of reproducibility-availability of data, materials, protocols, analysis scripts, and pre-registration-resulted in 2 indicators (availability of protocols and analysis scripts) being reproduced, as they fell within the 95% confidence intervals for the proportions from the original sample. However, materials' availability and pre-registration proportions from the validation sample were lower than what was found in the original sample.
conclusionOur findings demonstrate key indicators of reproducibility are missing in the field of radiology. Thus, the ability to reproduce studies contained in radiology publications may be problematic and may have potential clinical implications.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.