Evidence map›Paper›PMID 29233133›Full record

ArticleBMC medical research methodology2017

User testing of an adaptation of fishbone diagrams to depict results of systematic reviews.

Gerald Gartlehner, Marie-Therese Schultes, Viktoria Titscher, Laura C Morgan, Georgiy V Bobashev, Peyton Williams, Suzanne L West

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in BMC medical research methodology, 2017. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 4 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
4citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

4 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Gerald GartlehnerRTI International, 3040 East Cornwallis Rd, Research Triangle Park, Durham, NC, 27709, USA. ggartlehner@rti.org.
Marie-Therese SchultesDepartment for Evidence-based Medicine and Clinical Epidemiology, 3500, Krems, Austria.
Viktoria TitscherDepartment for Evidence-based Medicine and Clinical Epidemiology, 3500, Krems, Austria.
Laura C MorganRTI International, 3040 East Cornwallis Rd, Research Triangle Park, Durham, NC, 27709, USA.
Georgiy V BobashevRTI International, 3040 East Cornwallis Rd, Research Triangle Park, Durham, NC, 27709, USA.
Peyton WilliamsRTI International, 3040 East Cornwallis Rd, Research Triangle Park, Durham, NC, 27709, USA.
Suzanne L WestRTI International, 3040 East Cornwallis Rd, Research Triangle Park, Durham, NC, 27709, USA.

Funding

Predictive Models of Alcohol ConsumptionR03AA019775 · NIAAA · RESEARCH TRIANGLE INSTITUTE · PI BOBASHEV, GEORGIY · 2010 to 2011
$170k
NIAAA NIH HHS R03 AA019775
6 · The paper itself

Abstract

backgroundSummary of findings tables in systematic reviews are highly informative but require epidemiological training to be interpreted correctly. The usage of fishbone diagrams as graphical displays could offer researchers an effective approach to simplify content for readers with limited epidemiological training. In this paper we demonstrate how fishbone diagrams can be applied to systematic reviews and present the results of an initial user testing.

methodsFindings from two systematic reviews were graphically depicted in the form of the fishbone diagram. To test the utility of fishbone diagrams compared with summary of findings tables, we developed and pilot-tested an online survey using Qualtrics. Respondents were randomized to the fishbone diagram or a summary of findings table presenting the same body of evidence. They answered questions in both open-ended and closed-answer formats; all responses were anonymous. Measures of interest focused on first and second impressions, the ability to find and interpret critical information, as well as user experience with both displays. We asked respondents about the perceived utility of fishbone diagrams compared to summary of findings tables. We analyzed quantitative data by conducting t-tests and comparing descriptive statistics.

resultsBased on real world systematic reviews, we provide two different fishbone diagrams to show how they might be used to display complex information in a clear and succinct manner. User testing on 77 students with basic epidemiological training revealed that participants preferred summary of findings tables over fishbone diagrams. Significantly more participants liked the summary of findings table than the fishbone diagram (71.8% vs. 44.8%; p < .01); significantly more participants found the fishbone diagram confusing (63.2% vs. 35.9%, p < .05) or indicated that it was difficult to find information (65.8% vs. 45%; p < .01). However, more than half of the participants in both groups were unable to find critical information and answer three respective questions correctly (52.6% in the fishbone group; 51.3% in the summary of findings group).

conclusionsFishbone diagrams are compact visualizations that, theoretically, may prove useful for summarizing the findings of systematic reviews. Initial user testing, however, did not support the utility of such graphical displays.

Indexed as

Review Literature as TopicData Interpretation, StatisticalEvidence-Based MedicineHumansEvidence summaryFishbone diagramSummary of findingsSystematic reviewUser testingVisualization

Identifiers

PMID29233133
PMCPMC5727698

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.