ArticleBMJ open2024
Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis.
Article in BMJ open, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 6 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
6 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The proportion of people with a first episode of psychosis admitted to hospital at initial presentation: a systematic review and meta-analysis.Psychological medicine · 2025Pooled it
- GLP-1 Receptor Agonists in Metabolic Dysfunction-Associated Steatotic Liver Disease: Bridging Hepatic and Cardiovascular Outcomes.Chronic diseases and translational medicine · 2026Review
- Housing Instability as a Barrier to HIV Viral Suppression: Evidence from a U.S. Systematic Review and Meta-analysis.AIDS and behavior · 2026Review
- Extracorporeal COAnnals of intensive care · 2026Article
- Finding the right tool for the specific task: navigating RWE tools and checklists.Future oncology (London, England) · 2025Review
- The effect of an educational intervention for university students on violence against the elderly using gamification: a non-randomized clinical trial.Revista latino-americana de enfermagem · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectivesWe aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level.
designLiterature review through three approaches: systematic search of journal articles, snowballing search of reviews on appraisal tools and grey literature search on websites of health technology assessment (HTA) agencies. DATA SOURCES: Systematic search: Medline; Snowballing: starting from three articles (D'Andrea ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included a tool, if it addressed quality concerns of NRSIs and was published in English (unless from grey literature). A tool was excluded, if it was only for diagnostic, prognostic, qualitative or secondary studies. DATA EXTRACTION AND SYNTHESIS: Two independent researchers searched, screened and reviewed all included studies and tools, summarised quality items and scored whether and to what extent a quality item was described by a tool, for either methodological quality or reporting.
resultsForty-nine tools met inclusion criteria and were included for the content analysis. Concerns regarding the quality of NRSI were categorised into 4 domains and 26 items. The Research Triangle Institute Item Bank (RTI Item Bank) and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) were the most comprehensive tools for methodological quality and reporting, respectively, as they addressed (n=20; 17) and sufficiently described (n=18; 13) the highest number of items. However, none of the tools covered all items.
conclusionMost of the tools have their own strengths, but none of them could address all quality concerns relevant to NRSIs. Even the most comprehensive tools can be complemented by several items. We suggest decision-makers, researchers and tool developers consider the quality-item level heterogeneity, when selecting a tool or identifying a research gap. OSF REGISTRATION NUMBER: OSF registration DOI (https://doi.org/10.17605/OSF.IO/KCSGX).
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.