Evidence map›Paper›PMID 16477057›Full record

ArticleCMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne2006

Evidence of bias and variation in diagnostic accuracy studies.

Anne W S Rutjes, Johannes B Reitsma, Marcello Di Nisio, Nynke Smidt, Jeroen C van Rijn, Patrick M M Bossuyt

Registry-linked trialAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne, 2006. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT03116217 (Multicentre Validation of a Fetal Urine Peptidome-based Classifier to Predict Post-natal Renal Function in Posterior Urethral Valves), which is not on this map. Cited by 243 papers, 82 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
243citing papers in PubMed, 82 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT03116217 unknown statusnot on this mapstarted 2017, after this paper: background citation

Multicentre Validation of a Fetal Urine Peptidome-based Classifier to Predict Post-natal Renal Function in Posterior Urethral Valves

TypeobservationalSponsorUniversity Hospital, ToulouseRan2017 to 2023Enrolled400ConditionsPosterior Urethral Valves
3 · Its place in the literature

Who cites it

243 citing papers in PubMed, 82 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Nanopore sequencing for microbiological diagnosis of bacterial pneumonia: A systematic review and meta-analysis.European journal of clinical microbiology & infectious diseases : official publication of the European Society of Clinical Microbiology · 2026
    Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

183 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

6 authors.

Anne W S RutjesDeptartment of Clinical Epidemiology and Biostatistics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands.
Johannes B Reitsma
Marcello Di Nisio
Nynke Smidt
Jeroen C van Rijn
Patrick M M Bossuyt

Funding

HSRD VA SDR 10-012
6 · The paper itself

Abstract

backgroundStudies with methodologic shortcomings can overestimate the accuracy of a medical test. We sought to determine and compare the direction and magnitude of the effects of a number of potential sources of bias and variation in studies on estimates of diagnostic accuracy.

methodsWe identified meta-analyses of the diagnostic accuracy of tests through an electronic search of the databases MEDLINE, EMBASE, DARE and MEDION (1999-2002). We included meta-analyses with at least 10 primary studies without preselection based on design features. Pairs of reviewers independently extracted study characteristics and original data from the primary studies. We used a multivariable meta-epidemiologic regression model to investigate the direction and strength of the association between 15 study features on estimates of diagnostic accuracy.

resultsWe selected 31 meta-analyses with 487 primary studies of test evaluations. Only 1 study had no design deficiencies. The quality of reporting was poor in most of the studies. We found significantly higher estimates of diagnostic accuracy in studies with nonconsecutive inclusion of patients (relative diagnostic odds ratio [RDOR] 1.5, 95% confidence interval [CI] 1.0-2.1) and retrospective data collection (RDOR 1.6, 95% CI 1.1-2.2). The estimates were highest in studies that had severe cases and healthy controls (RDOR 4.9, 95% CI 0.6-37.3). Studies that selected patients based on whether they had been referred for the index test, rather than on clinical symptoms, produced significantly lower estimates of diagnostic accuracy (RDOR 0.5, 95% CI 0.3-0.9). The variance between meta-analyses of the effect of design features was large to moderate for type of design (cohort v. case-control), the use of composite reference standards and the use of differential verification; the variance was close to zero for the other design features.

interpretationShortcomings in study design can affect estimates of diagnostic accuracy, but the magnitude of the effect may vary from one situation to another. Design features and clinical characteristics of patient groups should be carefully considered by researchers when designing new studies and by readers when appraising the results of such studies. Unfortunately, incomplete reporting hampers the evaluation of potential sources of bias in diagnostic accuracy studies.

Indexed as

Clinical Trials as TopicDiagnosis, DifferentialDiagnostic Techniques and ProceduresHumansMeta-Analysis as TopicReproducibility of ResultsResearch Design

Identifiers

PMID16477057
PMCPMC1373751

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.