Evidence map›Paper›PMID 33937804›Full record

ArticleRadiology. Artificial intelligence2019

A User Interface for Optimizing Radiologist Engagement in Image Data Curation for Artificial Intelligence.

Mutlu Demirer, Sema Candemir, Matthew T Bigelow, Sarah M Yu, Vikash Gupta, Luciano M Prevedello, Richard D White, Joseph S Yu, Rainer Grimmer, Michael Wels and 5 more

Open access · bronzeAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Radiology. Artificial intelligence, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers.

0numbers the graph read from it
0cells of the map it votes in
18citing papers in PubMed
3.8field-weighted citation impact, top 6% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

18 citing papers in PubMed, 39 citations in OpenAlex.

  1. Article
  2. Article
  3. Review
  4. Article
  5. Article
  6. Article
  7. Review
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Review
  14. Article
  15. Article
  16. Blockchain in healthcare today · 2021
    Article
  17. Article
  18. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors at 2 institutions in 2 countries.

Mutlu DemirerDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0002-6842-282X
Sema CandemirDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).
Matthew T BigelowDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0002-4733-5911
Sarah M YuDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).
Vikash GuptaDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0002-6786-9865
Luciano M PrevedelloDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0002-6768-6452
Richard D WhiteDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).
Joseph S YuDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0001-6832-7243
Rainer GrimmerDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).
Michael WelsDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).
Andreas WimmerDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0003-1009-4306
Abdul H HalabiDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).
Alvin IhsaniDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).
Thomas P O'DonnellDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0003-3553-1306
Barbaros S ErdalDepartment of Radiology, Laboratory for Augmented Intelligence in Imaging-Division of Medical Imaging Informatics, Ohio State University College of Medicine, OSU Wexner Medical Center, 395 W 12th Ave, Suite 452, Columbus, OH 43210 (M.D., S.C., M.T.B., S.M.Y., V.G., L.M.P., R.D.W., J.S.Y., B.S.E.); Siemens Healthineers, Erlangen, Germany (R.G., M.W., A.W.); NVIDIA, Santa Clara, Calif (A.H.H., A.I.); and Siemens Healthineers, Malvern, Pa (T.P.O.).ORCID 0000-0003-3637-0102
The Ohio State University Wexner Medical Center · USSiemens Healthcare (United States) · US

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeTo delineate image data curation needs and describe a locally designed graphical user interface (GUI) to aid radiologists in image annotation for artificial intelligence (AI) applications in medical imaging. MATERIALS AND

methodsGUI components support image analysis toolboxes, picture archiving and communication system integration, third-party applications, processing of scripting languages, and integration of deep learning libraries. For clinical AI applications, GUI components included two-dimensional segmentation and classification; three-dimensional segmentation and quantification; and three-dimensional segmentation, quantification, and classification. To assess radiologist engagement and performance efficiency associated with GUI-related capabilities, image annotation rate (studies per day) and speed (minutes per case) were evaluated in two clinical scenarios of varying complexity: hip fracture detection and coronary atherosclerotic plaque demarcation and stenosis grading.

resultsFor hip fracture, 1050 radiographs were annotated over 7 days (150 studies per day; median speed: 10 seconds per study [interquartile range, 3-21 seconds per study]). A total of 294 coronary CT angiographic studies with 1843 arteries and branches were annotated for atherosclerotic plaque over 23 days (15.2 studies [80.1 vessels] per day; median speed: 6.08 minutes per study [interquartile range, 2.8-10.6 minutes per study] and 73 seconds per vessel [interquartile range, 20.9-155 seconds per vessel]).

conclusionGUI-component compatibility with common image analysis tools facilitates radiologist engagement in image data curation, including image annotation, supporting AI application development and evolution for medical imaging. When complemented by other GUI elements, a continuous integrated workflow supporting formation of an agile deep neural network life cycle results.Supplemental material is available for this article.© RSNA, 2019.

Identifiers

PMID33937804
PMCPMC8017380
OpenAlexW2991634334

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.