Evidence map›Paper›PMID 38961010›Full record

ReviewQuality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation2024

Methodological quality of 100 recent systematic reviews of health-related outcome measurement instruments: an overview of reviews.

Ellen B M Elsman, Lidwine B Mokkink, Inger L Abma, Olalekan Lee Aiyegbusi, Alessandro Chiarotto, Kirstie L Haywood, Karen Matvienko-Sikar, Daniella M Oosterveer, Jan J M Pool, Ilse E J Swinkels-Meewisse and 2 more

Abstract readReview
In one paragraph

Review in Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers, 3 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
8citing papers in PubMed, 3 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

8 citing papers in PubMed, 3 syntheses or guidelines pooled it.

  1. Pooled it
  2. Evaluating the psychometric properties of patient-reported outcome measurements for urolithiasis: a COSMIN-based systematic review.Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2025
    Pooled it
  3. Pooled it
  4. The COSMIN Clubs initiative: improving health measurement through collaborative learning.Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2026
    Article
  5. Assessing content validity: challenges of conducting systematic reviews of patient-reported outcome measures and recommendations to improve the application of COSMIN guidance.Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2026
    Article
  6. Review
  7. Systematic reviews of patient-reported outcome measures (PROMs): table templates for effective communication.Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2025
    Article
  8. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

12 authors.

Ellen B M ElsmanDepartment of Epidemiology & Data Science, Amsterdam UMC, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, De Boelelaan 1089a, 1081 HV, Amsterdam, The Netherlands.ORCID http://orcid.org/0000-0002-0680-6430
Lidwine B MokkinkDepartment of Epidemiology & Data Science, Amsterdam UMC, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, De Boelelaan 1089a, 1081 HV, Amsterdam, The Netherlands.ORCID http://orcid.org/0000-0001-6489-2827
Inger L AbmaIQ Health, Radboud Institute of Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands.ORCID http://orcid.org/0000-0002-3307-6736
Olalekan Lee AiyegbusiCentre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.ORCID http://orcid.org/0000-0001-9122-8251
Alessandro ChiarottoDepartment of General Practice, Erasmus MC, University Medical Center, Rotterdam, The Netherlands.ORCID http://orcid.org/0000-0003-2350-9945
Kirstie L HaywoodWarwick Applied Health, Warwick Medical School, University of Warwick, Coventry, UK.ORCID http://orcid.org/0000-0002-5405-187X
Karen Matvienko-SikarSchool of Public Health, University College Cork, Cork, Ireland.ORCID http://orcid.org/0000-0003-2777-6581
Daniella M OosterveerBasalt, Leiden/The Hague, The Netherlands.ORCID http://orcid.org/0000-0003-1540-6903
Jan J M PoolUniversity of Applied Sciences, Utrecht, The Netherlands.ORCID http://orcid.org/0000-0002-9240-4488
Ilse E J Swinkels-MeewisseAmsterdam UMC, Amsterdam, The Netherlands.
Martin OffringaChild Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, ON, Canada.ORCID http://orcid.org/0000-0002-4402-5299
Caroline B TerweeDepartment of Epidemiology & Data Science, Amsterdam UMC, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, De Boelelaan 1089a, 1081 HV, Amsterdam, The Netherlands. cb.terwee@amsterdamumc.nl.ORCID http://orcid.org/0000-0003-4570-2826

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeSystematic reviews evaluating and comparing the measurement properties of outcome measurement instruments (OMIs) play an important role in OMI selection. Earlier overviews of review quality (2007, 2014) evidenced substantial concerns with regards to alignment to scientific standards. This overview aimed to investigate whether the quality of recent systematic reviews of OMIs lives up to the current scientific standards.

methodsOne hundred systematic reviews of OMIs published from June 1, 2021 onwards were randomly selected through a systematic literature search performed on March 17, 2022 in MEDLINE and EMBASE. The quality of systematic reviews was appraised by two independent reviewers. An updated data extraction form was informed by the earlier studies, and results were compared to these earlier studies' findings.

resultsA quarter of the reviews had an unclear research question or aim, and in 22% of the reviews the search strategy did not match the aim. Half of the reviews had an incomprehensive search strategy, because relevant search terms were not included. In 63% of the reviews (compared to 41% in 2014 and 30% in 2007) a risk of bias assessment was conducted. In 73% of the reviews (some) measurement properties were evaluated (58% in 2014 and 55% in 2007). In 60% of the reviews the data were (partly) synthesized (42% in 2014 and 7% in 2007); evaluation of measurement properties and data syntheses was not conducted separately for subscales in the majority. Certainty assessments of the quality of the total body of evidence were conducted in only 33% of reviews (not assessed in 2014 and 2007). The majority (58%) did not make any recommendations on which OMI (not) to use.

conclusionDespite clear improvements in risk of bias assessments, measurement property evaluation and data synthesis, specifying the research question, conducting the search strategy and performing a certainty assessment remain poor. To ensure that systematic reviews of OMIs meet current scientific standards, more consistent conduct and reporting of systematic reviews of OMIs is needed.

Indexed as

Outcome Assessment, Health CareSystematic Reviews as TopicHumansCOSMINMeasurement propertiesOutcome measurement instrumentsReliabilitySystematic reviewsValidity

Identifiers

PMID38961010
PMCPMC11452433

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.