ArticleJournal of the American Medical Informatics Association : JAMIA2019
Criteria2Query: a natural language interface to clinical databases for cohort definition.
Article in Journal of the American Medical Informatics Association : JAMIA, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 86 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
86 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- A systematic review on natural language processing systems for eligibility prescreening in clinical research.Journal of the American Medical Informatics Association : JAMIA · 2021Pooled it
- Clinical Trial Generalizability Assessment in the Big Data Era: A Review.Clinical and translational science · 2020Pooled it
- Leveraging electronic health records for atrial fibrillation cohort generation.Health information science and systems · 2026Article
- Artificial intelligence in clinical trials-state of the evidence, gaps, and next steps.EClinicalMedicine · 2026Review
- Health Care Analytics Challenges: A 3-Pillar Framework Connecting Analytics Maturity, Workforce Agility, and Technical Enablement.Interactive journal of medical research · 2026Article
- A Natural Language Processing Framework for Structuring and Visualizing Clinical Trial Eligibility Criteria at Scale: Protocol for a Quantitative Study.JMIR research protocols · 2026Article
- Article
- TrialMatchAI: an end-to-end AI-powered clinical trial recommendation system to streamline patient-to-trial matching.Nature communications · 2026Article
- Transformer-based relation extraction and concept normalization using an annotated clinical trials corpus.Scientific data · 2026Article
- AI in the Hot Seat: Head-to-Head Comparison of Large Language Models and Cardiologists in Emergency Scenarios.Medical sciences (Basel, Switzerland) · 2026Article
- Artificial intelligence in clinical trial participant recruitment and retention: A scoping review and meta-analysis.Journal of clinical and translational science · 2026Review
- EC2Seq2Sql: Patient-trial matching with LLM agents.PloS one · 2026Article
- Completeness of Common Data Elements for Breast Cancer Clinical Trials in Observational Databases.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2026Article
- AI-augmented frameworks for enhancing Alzheimer's disease clinical trials: A memory clinic perspective.The journal of prevention of Alzheimer's disease · 2025Article
- Large Language Models for Automating Clinical Trial Criteria Conversion to Observational Medical Outcomes Partnership Common Data Model Queries: Validation and Evaluation Study.JMIR medical informatics · 2025Article
- CLEAR: A vision to support clinical evidence lifecycle with continuous learning.Journal of biomedical informatics · 2025Article
- A review on knowledge graphs for healthcare: Resources, applications, and promises.Journal of biomedical informatics · 2025Review
- Enhancing Patient-Trial Matching With Large Language Models: A Scoping Review of Emerging Applications and Approaches.JCO clinical cancer informatics · 2025Article
- Analysis of eligibility criteria clusters based on large language models for clinical trial design.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Hybrid natural language processing tool for semantic annotation of medical texts in Spanish.BMC bioinformatics · 2025Article
26 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
Abstract
objectiveCohort definition is a bottleneck for conducting clinical research and depends on subjective decisions by domain experts. Data-driven cohort definition is appealing but requires substantial knowledge of terminologies and clinical data models. Criteria2Query is a natural language interface that facilitates human-computer collaboration for cohort definition and execution using clinical databases. MATERIALS AND
methodsCriteria2Query uses a hybrid information extraction pipeline combining machine learning and rule-based methods to systematically parse eligibility criteria text, transforms it first into a structured criteria representation and next into sharable and executable clinical data queries represented as SQL queries conforming to the OMOP Common Data Model. Users can interactively review, refine, and execute queries in the ATLAS web application. To test effectiveness, we evaluated 125 criteria across different disease domains from ClinicalTrials.gov and 52 user-entered criteria. We evaluated F1 score and accuracy against 2 domain experts and calculated the average computation time for fully automated query formulation. We conducted an anonymous survey evaluating usability.
resultsCriteria2Query achieved 0.795 and 0.805 F1 score for entity recognition and relation extraction, respectively. Accuracies for negation detection, logic detection, entity normalization, and attribute normalization were 0.984, 0.864, 0.514 and 0.793, respectively. Fully automatic query formulation took 1.22 seconds/criterion. More than 80% (11+ of 13) of users would use Criteria2Query in their future cohort definition tasks.
conclusionsWe contribute a novel natural language interface to clinical databases. It is open source and supports fully automated and interactive modes for autonomous data-driven cohort definition by researchers with minimal human effort. We demonstrate its promising user friendliness and usability.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.