ArticleNPJ digital medicine2021
Biological data annotation via a human-augmenting AI-based labeling system.
Article in NPJ digital medicine, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed, 38 citations in OpenAlex.
- A scoping review of human-AI collaboration patterns and task divisions in healthcare applications.NPJ digital medicine · 2026Article
- Deep Learning-Based Tumor Cell Classification in Lung Adenocarcinoma With a Case-by-Case Human-in-the-Loop Approach.APMIS : acta pathologica, microbiologica, et immunologica Scandinavica · 2026Article
- EpiCure (Epithelial Curation): a versatile and handy tool for curation of epithelial segmentation.bioRxiv : the preprint server for biology · 2026Article
- A review of image processing and analysis of computed tomography images using deep learning methods.Physical and engineering sciences in medicine · 2025Review
- A conceptual framework for human-AI collaborative genome annotation.Briefings in bioinformatics · 2025Review
- A pathologist-AI collaboration framework for enhancing diagnostic accuracies and efficiencies.Nature biomedical engineering · 2025Article
- Boosting wisdom of the crowd for medical image annotation using training performance and task features.Cognitive research: principles and implications · 2024Article
- Variational Autoencoders for Biomedical Signal Morphology Clustering and Noise Detection.IEEE journal of biomedical and health informatics · 2023Article
- From function to translation: Decoding genetic susceptibility to human diseases via artificial intelligence.Cell genomics · 2023Review
- Which data subset should be augmented for deep learning? a simulation study using urothelial cell carcinoma histopathology images.BMC bioinformatics · 2023Article
- Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study.JMIR medical informatics · 2023Article
- Deep Interactive Learning-based ovarian cancer segmentation of H&E-stained whole slide images to study morphological patterns of BRCA mutation.Journal of pathology informatics · 2023Article
- Semantic annotation for computational pathology: multidisciplinary experience and best practice recommendations.The journal of pathology. Clinical research · 2022Article
- Efficient cellular annotation of histopathology slides with real-time AI augmentation.NPJ digital medicine · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
14 authors at 4 institutions in 2 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Biology has become a prime area for the deployment of deep learning and artificial intelligence (AI), enabled largely by the massive data sets that the field can generate. Key to most AI tasks is the availability of a sufficiently large, labeled data set with which to train AI models. In the context of microscopy, it is easy to generate image data sets containing millions of cells and structures. However, it is challenging to obtain large-scale high-quality annotations for AI models. Here, we present HALS (Human-Augmenting Labeling System), a human-in-the-loop data labeling AI, which begins uninitialized and learns annotations from a human, in real-time. Using a multi-part AI composed of three deep learning models, HALS learns from just a few examples and immediately decreases the workload of the annotator, while increasing the quality of their annotations. Using a highly repetitive use-case-annotating cell types-and running experiments with seven pathologists-experts at the microscopic analysis of biological specimens-we demonstrate a manual work reduction of 90.60%, and an average data-quality boost of 4.34%, measured across four use-cases and two tissue stain types.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.