Evidence map›Paper›PMID 24713692›Full record

SynthesisMedical decision making : an international journal of the Society for Medical Decision Making2014

"It's Valid and Reliable" Is Not Enough: Critical Appraisal of Reporting of Measures in Trials Evaluating Patient Decision Aids.

Karen R Sepucha, Daniel D Matlock, Celia E Wills, Mary Ropka, Natalie Joseph-Williams, Dawn Stacey, ChirkJenn Ng, Carrie Levin, Joanne Lally, Cornelia M Borkhoff and 1 more

Abstract readSystematic Review
In one paragraph

Synthesis in Medical decision making : an international journal of the Society for Medical Decision Making, 2014. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 3 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
17citing papers in PubMed, 3 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

17 citing papers in PubMed, 3 syntheses or guidelines pooled it.

  1. Operationalizing and Measuring Informed Choice in Health Care: An Umbrella Review.Medical decision making : an international journal of the Society for Medical Decision Making · 2026
    Pooled it
  2. Pooled it
  3. Are We Improving? Update and Critical Appraisal of the Reporting of Decision Process and Quality Measures in Trials Evaluating Patient Decision Aids.Medical decision making : an international journal of the Society for Medical Decision Making · 2021
    Pooled it
  4. Review
  5. Article
  6. Article
  7. Article
  8. Development and Psychometric Testing of EPAT-16: A Short and Valid Measure for Patient-Centeredness From the Patient's Perspective.Health expectations : an international journal of public participation in health care and health policy · 2025
    Article
  9. Translation and Psychometric Evaluation in Cancer Care of the German Version of collaboRATEHealth expectations : an international journal of public participation in health care and health policy · 2025
    Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Review
  17. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

11 authors.

Karen R SepuchaGeneral Medicine Division, Massachusetts General Hospital and Harvard Medical School, Boston, MA (KS)
Daniel D MatlockUniversity of Colorado School of Medicine, Aurora, CO (DDM)The Ohio State University, Columbus, OH (CEW)
Celia E WillsThe Ohio State University, Columbus, OH (CEW)
Mary RopkaDepartment of Public Health Sciences, University of Virginia School of Medicine, Charlottesville, VA (MR)
Natalie Joseph-WilliamsCochrane Institute of Primary Care and Public Health, Cardiff University, CARDIFF, UK (NJ)
Dawn StaceyUniversity of Ottawa and Ottawa Hospital Research Institute, Ottawa, Canada (DS)
ChirkJenn NgDepartment of Primary Care Medicine, University of Malaya, Kuala Lumpur, Malaya (CN)
Carrie LevinInformed Medical Decisions Foundation, Boston, MA (CL)
Joanne LallyInstitute of Health and Society, Newcastle University, Newcastle, UK (JL, RT)
Cornelia M BorkhoffUniversity of Toronto and The Hospital for Sick Children, Toronto, ON, Canada (CMB).
Richard ThomsonInstitute of Health and Society, Newcastle University, Newcastle, UK (JL, RT)

Funding

Implanted Defibrillators and Older Adults: A Model of Decisions and TechnologiesK23AG040696 · NIA · UNIVERSITY OF COLORADO DENVER · PI MATLOCK, DANIEL D · 2011 to 2015
$716k
NIA NIH HHS K23 AG040696
6 · The paper itself

Abstract

backgroundThis review systematically appraises the quality of reporting of measures used in trials to evaluate the effectiveness of patient decision aids (PtDAs) and presents recommendations for minimum reporting standards.

methodsWe reviewed measures of decision quality and decision process in 86 randomized controlled trials (RCTs) from the 2011 Cochrane Collaboration systematic review of PtDAs. Data on development of the measures, reliability, validity, responsiveness, precision, interpretability, feasibility, and acceptability were independently abstracted by 2 reviewers.

resultsInformation from 178 instances of use of measures was abstracted. Very few studies reported data on the performance of measures, with reliability (21%) and validity (16%) being the most common. Studies using new measures were less likely to include information about their psychometric performance. The review was limited to reporting of measures in studies included in the Cochrane review and did not consult prior publications.

conclusionsVery little is reported about the development or performance of measures used to evaluate the effectiveness of PtDAs in published trials. Minimum reporting standards are proposed to enable authors to prepare study reports, editors and reviewers to evaluate submitted papers, and readers to appraise published studies.

Indexed as

Decision Support TechniquesData AccuracyHumansPatient ParticipationPsychometricsReproducibility of ResultsResearch Designdecision aidsdecision makingpsychometrics

Identifiers

PMID24713692
PMCPMC4190105

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.