Evidence map›Paper›PMID 34271832›Full record

ArticleMedical decision making : an international journal of the Society for Medical Decision Making2022

Reference Case Methods for Expert Elicitation in Health Care Decision Making.

Laura Bojke, Marta O Soares, Karl Claxton, Abigail Colson, Aimée Fox, Chris Jackson, Dina Jankovic, Alec Morton, Linda D Sharples, Andrea Taylor

Abstract read
In one paragraph

Article in Medical decision making : an international journal of the Society for Medical Decision Making, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 29 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
29citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

29 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. The Use of Expert Elicitation among Computational Modeling Studies in Health Research: A Systematic Review.Medical decision making : an international journal of the Society for Medical Decision Making · 2022
    Pooled it
  3. Trial
  4. General-Purpose Methods for Simulating Survival Data for Expected Value of Sample Information Calculations.Medical decision making : an international journal of the Society for Medical Decision Making · 2023
    Trial
  5. Article
  6. Article
  7. Improving the use of expert opinion in disease risk analysis for conservation translocations.Conservation biology : the journal of the Society for Conservation Biology · 2026
    Article
  8. Article
  9. Article
  10. Article
  11. Review
  12. Co-designing a Structured Expert Elicitation with Clinicians to Enhance Health Care Decision Making in Exercise Oncology.Medical decision making : an international journal of the Society for Medical Decision Making · 2025
    Article
  13. Article
  14. Development of the PICCOTEAM Reference Case for Economic Evaluation of Precision Medicine.International journal of health policy and management · 2025
    Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Methods to Quantify the Importance of Parameters for Model Updating and Distributional Adaptation.Medical decision making : an international journal of the Society for Medical Decision Making · 2024
    Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Laura BojkeCentre for Health Economics, University of York, York, UK.ORCID 0000-0001-7921-9109
Marta O SoaresCentre for Health Economics, University of York, York, UK.ORCID 0000-0003-1579-8513
Karl ClaxtonCentre for Health Economics, University of York, York, UK.
Abigail ColsonThe Department of Management Science, University of Strathclyde, Glasgow, UK.ORCID 0000-0002-3241-5855
Aimée FoxCentre for Health Economics, University of York, York, UK.ORCID 0000-0001-6944-7554
Chris JacksonMRC Biostatistics Unit, University of Cambridge, Cambridge, UK.ORCID 0000-0002-6656-8913
Dina JankovicCentre for Health Economics, University of York, York, UK.ORCID 0000-0002-9311-1409
Alec MortonThe Department of Management Science, University of Strathclyde, Glasgow, UK.ORCID 0000-0003-3803-8517
Linda D SharplesLondon School of Hygiene and Tropical Medicine, London, UK.ORCID 0000-0003-0894-966X
Andrea TaylorLeeds University Business School, Leeds, UK.

Funding

Medical Research Council MC_UU_00002/11Medical Research Council MR/N028511/1
6 · The paper itself

Abstract

backgroundThe evidence used to inform health care decision making (HCDM) is typically uncertain. In these situations, the experience of experts is essential to help decision makers reach a decision. Structured expert elicitation (referred to as elicitation) is a quantitative process to capture experts' beliefs. There is heterogeneity in the existing elicitation methodology used in HCDM, and it is not clear if existing guidelines are appropriate for use in this context. In this article, we seek to establish reference case methods for elicitation to inform HCDM.

methodsWe collated the methods available for elicitation using reviews and critique. In addition, we conducted controlled experiments to test the accuracy of alternative methods. We determined the suitability of the methods choices for use in HCDM according to a predefined set of principles for elicitation in HCDM, which we have also generated. We determined reference case methods for elicitation in HCDM for health technology assessment (HTA).

resultsIn almost all methods choices available for elicitation, we found a lack of empirical evidence supporting recommendations. Despite this, it is possible to define reference case methods for HTA. The reference methods include a focus on gathering experts with substantive knowledge of the quantities being elicited as opposed to those trained in probability and statistics, eliciting quantities that the expert might observe directly, and individual elicitation of beliefs, rather than solely consensus methods. It is likely that there are additional considerations for decision makers in health care outside of HTA.

conclusionsThe reference case developed here allows the use of different methods, depending on the decision-making setting. Further applied examples of elicitation methods would be useful. Experimental evidence comparing methods should be generated.

Indexed as

Expert TestimonyTechnology Assessment, BiomedicalDecision MakingDelivery of Health CareHumansProbabilityUncertaintyDecision-makingeconomic-evaluationElicitationuncertainty

Identifiers

PMID34271832
PMCPMC8777312

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.