Evidence map›Paper›PMID 32725679›Full record

ReviewNephrology (Carlton, Vic.)2021

Funnel plots of patient-reported outcomes to evaluate health-care quality: Basic principles, pitfalls and considerations.

Esmee M van der Willik, Erik W van Zwet, Tiny Hoekstra, Frans J van Ittersum, Marc H Hemmelder, Carmine Zoccali, Kitty J Jager, Friedo W Dekker, Yvette Meuleman

Abstract readReview
In one paragraph

Review in Nephrology (Carlton, Vic.), 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
16citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

16 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Assessing the real-world value of hemophilia care: how suitable are patient-centered outcomes for hospital-level comparisons?International journal for quality in health care : journal of the International Society for Quality in Health Care · 2026
    Article
  4. Article
  5. Article
  6. Review
  7. Article
  8. A method to identify prescription drug targets for health technology reassessment.International journal of technology assessment in health care · 2025
    Article
  9. Article
  10. Article
  11. Article
  12. Review
  13. Funnel plots a graphical instrument for the evaluation of population performance and quality of trauma care: a blueprint of implementation.European journal of trauma and emergency surgery : official publication of the European Trauma Society · 2023
    Observational
  14. Article
  15. Article
  16. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors.

Esmee M van der WillikDepartment of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands.ORCID https://orcid.org/0000-0001-9457-5857
Erik W van ZwetDepartment of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands.
Tiny HoekstraDepartment of Nephrology, Amsterdam University Medical Centre, Amsterdam, The Netherlands.
Frans J van IttersumDepartment of Nephrology, Amsterdam University Medical Centre, Amsterdam, The Netherlands.
Marc H HemmelderNefrovisie Foundation, Utrecht, The Netherlands.
Carmine ZoccaliCNR-IFC, Clinical Epidemiology and Physiopathology of Renal Diseases and Hypertension, Reggio Calabria, Italy.
Kitty J JagerERA-EDTA Registry, Department of Medical Informatics, Amsterdam UMC, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.
Friedo W DekkerDepartment of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands.
Yvette MeulemanDepartment of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

A funnel plot is a graphical method to evaluate health-care quality by comparing hospital performances on certain outcomes. So far, in nephrology, this method has been applied to clinical outcomes like mortality and complications. However, patient-reported outcomes (PROs; eg, health-related quality of life [HRQOL]) are becoming increasingly important and should be incorporated into this quality assessment. Using funnel plots has several advantages, including clearly visualized precision, detection of volume-effects, discouragement of ranking hospitals and easy interpretation of results. However, without sufficient knowledge of underlying methods, it is easy to stumble into pitfalls, such as overinterpretation of standardized scores, incorrect direct comparisons of hospitals and assuming a hospital to be in-control (ie, to perform as expected) based on underpowered comparisons. Furthermore, application of funnel plots to PROs is accompanied by additional challenges related to the multidimensional nature of PROs and difficulties with measuring PROs. Before using funnel plots for PROs, high and consistent response rates, adequate case mix correction and high-quality PRO measures are required. In this article, we aim to provide insight into the use and interpretation of funnel plots by presenting an overview of the basic principles, pitfalls and considerations when applied to PROs, using examples from Dutch routine dialysis care.

Indexed as

Health Services ResearchNephrologyPatient Reported Outcome MeasuresQuality Assurance, Health CareQuality Indicators, Health CareResearch DesignBenchmarkingData Interpretation, StatisticalHumansModels, Statisticalbenchmarkingcase mix adjustmentmethodsnephrologypatient-reported outcomesquality of health care

Identifiers

PMID32725679
PMCPMC7891340

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.