ArticleJournal of the American Medical Informatics Association : JAMIA2019
Cohort selection for clinical trials: n2c2 2018 shared task track 1.
Article in Journal of the American Medical Informatics Association : JAMIA, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 36 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
36 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review.Frontiers in human neuroscience · 2023Pooled it
- Leveraging electronic health records for atrial fibrillation cohort generation.Health information science and systems · 2026Article
- The word and the way: strategies for domain-specific BERT pre-training in German medical NLP.BMC medical informatics and decision making · 2026Article
- Assessing genetic counseling efficiency with natural language processing.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Real-world validation of a multimodal LLM-powered pipeline for high-accuracy clinical trial patient matching.Communications medicine · 2025Article
- Towards Efficient Patient Recruitment for Clinical Trials: Application of a Prompt-Based Learning Model.Healthcare informatics research · 2025Article
- Enhancing Patient-Trial Matching With Large Language Models: A Scoping Review of Emerging Applications and Approaches.JCO clinical cancer informatics · 2025Article
- Synthetic data distillation enables the extraction of clinical information at scale.NPJ digital medicine · 2025Article
- Article
- Matching patients to clinical trials with large language models.Nature communications · 2024Article
- Transformers and large language models in healthcare: A review.Artificial intelligence in medicine · 2024Review
- Utilizing Large Language Models for Enhanced Clinical Trial Matching: A Study on Automation in Patient Screening.Cureus · 2024Article
- Article
- Clinical research staff perceptions on a natural language processing-driven tool for eligibility prescreening: An iterative usability assessment.International journal of medical informatics · 2023Article
- Machine learning approaches for electronic health records phenotyping: a methodical review.Journal of the American Medical Informatics Association : JAMIA · 2023Article
- Article
- Identify diabetic retinopathy-related clinical concepts and their attributes using transformer-based natural language processing methods.BMC medical informatics and decision making · 2022Article
- A scoping review of publicly available language tasks in clinical natural language processing.Journal of the American Medical Informatics Association : JAMIA · 2022Article
- Combining human and machine intelligence for clinical trial eligibility querying.Journal of the American Medical Informatics Association : JAMIA · 2022Article
- HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
objectiveTrack 1 of the 2018 National NLP Clinical Challenges shared tasks focused on identifying which patients in a corpus of longitudinal medical records meet and do not meet identified selection criteria. MATERIALS AND
methodsTo address this challenge, we annotated American English clinical narratives for 288 patients according to whether they met these criteria. We chose criteria from existing clinical trials that represented a variety of natural language processing tasks, including concept extraction, temporal reasoning, and inference.
resultsA total of 47 teams participated in this shared task, with 224 participants in total. The participants represented 18 countries, and the teams submitted 109 total system outputs. The best-performing system achieved a micro F1 score of 0.91 using a rule-based approach. The top 10 teams used rule-based and hybrid systems to approach the problems. DISCUSSION: Clinical narratives are open to interpretation, particularly in cases where the selection criterion may be underspecified. This leaves room for annotators to use domain knowledge and intuition in selecting patients, which may lead to error in system outputs. However, teams who consulted medical professionals while building their systems were more likely to have high recall for patients, which is preferable for patient selection systems.
conclusionsThere is not yet a 1-size-fits-all solution for natural language processing systems approaching this task. Future research in this area can look to examining criteria requiring even more complex inferences, temporal reasoning, and domain knowledge.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.