Evidence map›Paper›PMID 39702575›Full record

ArticleNPJ digital medicine2024

Optimising the paradigms of human AI collaborative clinical coding.

Yue Gao, Yuepeng Chen, Minghao Wang, Jinge Wu, Yunsoo Kim, Kaiyin Zhou, Miao Li, Xien Liu, Xiangling Fu, Ji Wu and 1 more

Abstract read
In one paragraph

Article in NPJ digital medicine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
8citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

8 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. Article
  5. Article
  6. Review
  7. Review
  8. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

11 authors.

Yue GaoSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China.
Yuepeng ChenSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China.
Minghao WangSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China.
Jinge WuInstitute of Health Informatics, University College London, London, UK.
Yunsoo KimInstitute of Health Informatics, University College London, London, UK.
Kaiyin ZhouSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China.
Miao LiDepartment of Electronic Engineering, Tsinghua University, Beijing, China.ORCID http://orcid.org/0009-0009-5672-7448
Xien LiuDepartment of Electronic Engineering, Tsinghua University, Beijing, China.
Xiangling FuSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China. fuxiangling@bupt.edu.cn.
Ji WuDepartment of Electronic Engineering, Tsinghua University, Beijing, China.
Honghan WuInstitute of Health Informatics, University College London, London, UK. honghan.wu@glasgow.ac.uk.ORCID http://orcid.org/0000-0002-0213-5668

Funding

DH | National Institute for Health Research (NIHR) NIHR202639National Natural Science Foundation of China (National Science Foundation of China) 72274022RCUK | MRC | Medical Research Foundation MR/X030075/1
6 · The paper itself

Abstract

Automated clinical coding (ACC) has emerged as a promising alternative to manual coding. This study proposes a novel human-in-the-loop (HITL) framework, CliniCoCo. Using deep learning capacities, CliniCoCo focuses on how such ACC systems and human coders can work effectively and efficiently together in real-world settings. Specifically, it implements a series of collaborative strategies at annotation, training and user interaction stages. Extensive experiments are conducted using real-world EMR datasets from Chinese hospitals. With automatically optimised annotation workloads, the model can achieve F1 scores around 0.80-0.84. For an EMR with 30% mistaken codes, CliniCoCo can suggest halving the annotations from 3000 admissions with an ignorable 0.01 F1 decrease. In human evaluations, compared to manual coding, CliniCoCo reduces coding time by 40% on average and significantly improves the correction rates on EMR mistakes (e.g., three times better on missing codes). Senior professional coders' performances can be boosted to more than 0.93 F1 score from 0.72.

Identifiers

PMID39702575
PMCPMC11659570

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.