Evidence map›Paper›PMID 27535658›Full record

SynthesisSystematic reviews2016

Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews.

Ian Shemilt, Nada Khan, Sophie Park, James Thomas

Abstract readSystematic Review
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in Systematic reviews, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 59 papers, 13 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
59citing papers in PubMed, 13 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

59 citing papers in PubMed, 13 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Reducing waste in collection of quality-of-life data through better reporting: a case study.Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2022
    Pooled it
  8. Pooled it
  9. Pooled it
  10. Guideline
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors.

Ian ShemiltSocial Sciences Research Unit, UCL Institute of Education, London, UK. i.shemilt@ucl.ac.uk.
Nada KhanDepartment of Primary Care and Population Health, University College London, London, UK.
Sophie ParkDepartment of Primary Care and Population Health, University College London, London, UK.
James ThomasSocial Sciences Research Unit, UCL Institute of Education, London, UK.

Funding

Medical Research Council MR/J005037/1
6 · The paper itself

Abstract

backgroundMeta-research studies investigating methods, systems, and processes designed to improve the efficiency of systematic review workflows can contribute to building an evidence base that can help to increase value and reduce waste in research. This study demonstrates the use of an economic evaluation framework to compare the costs and effects of four variant approaches to identifying eligible studies for consideration in systematic reviews.

methodsA cost-effectiveness analysis was conducted using a basic decision-analytic model, to compare the relative efficiency of 'safety first', 'double screening', 'single screening' and 'single screening with text mining' approaches in the title-abstract screening stage of a 'case study' systematic review about undergraduate medical education in UK general practice settings. Incremental cost-effectiveness ratios (ICERs) were calculated as the 'incremental cost per citation 'saved' from inappropriate exclusion' from the review. Resource use and effect parameters were estimated based on retrospective analysis of 'review process' meta-data curated alongside the 'case study' review, in conjunction with retrospective simulation studies to model the integrated use of text mining. Unit cost parameters were estimated based on the 'case study' review's project budget. A base case analysis was conducted, with deterministic sensitivity analyses to investigate the impact of variations in values of key parameters.

resultsUse of 'single screening with text mining' would have resulted in title-abstract screening workload reductions (base case analysis) of >60 % compared with other approaches. Across modelled scenarios, the 'safety first' approach was, consistently, equally effective and less costly than conventional 'double screening'. Compared with 'single screening with text mining', estimated ICERs for the two non-dominated approaches (base case analyses) ranged from £1975 ('single screening' without a 'provisionally included' code) to £4427 ('safety first' with a 'provisionally included' code) per citation 'saved'. Patterns of results were consistent between base case and sensitivity analyses.

conclusionsAlternatives to the conventional 'double screening' approach, integrating text mining, warrant further consideration as potentially more efficient approaches to identifying eligible studies for systematic reviews. Comparable economic evaluations conducted using other systematic review datasets are needed to determine the generalisability of these findings and to build an evidence base to inform guidance for review authors.

Indexed as

Cost-Benefit AnalysisResearch DesignData MiningHumansPatient Safety

Identifiers

PMID27535658
PMCPMC4989498

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.