ReviewJournal of the American Medical Informatics Association : JAMIA
Evaluating temporal relations in clinical text: 2012 i2b2 Challenge.
Review in Journal of the American Medical Informatics Association : JAMIA. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 173 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
173 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Discovery of Noncancer Drug Effects on Survival in Electronic Health Records of Patients With Cancer: A New Paradigm for Drug Repurposing.JCO clinical cancer informatics · 2019Pooled it
- Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review.Journal of the American Medical Informatics Association : JAMIA · 2019Pooled it
- Natural language processing systems for capturing and standardizing unstructured clinical information: A systematic review.Journal of biomedical informatics · 2017Pooled it
- Optimizing Clinical Temporal Relation Extraction With Large Language Models: Comparative Analysis.JMIR medical informatics · 2026Article
- Leveraging Generative Large Language Models for Temporal Relation Extraction From French Clinical Narratives: Prompt-Chaining Approach.Journal of medical Internet research · 2026Article
- Computable Phenotype for Identifying Undiagnosed Hypermobile Ehlers-Danlos Syndrome: Protocol for a Development and Validation Study.JMIR research protocols · 2026Article
- A study of large language models for patient information extraction: Model architecture, fine-tuning strategy, and multi-task instruction tuning.Journal of biomedical informatics · 2026Article
- From annotation to adaptation: extracting temporal relations in French clinical narratives.BMC medical research methodology · 2026Article
- Article
- Ontology-Based Medication Named Entity Recognition Using Pretrained Transformer Models From a Thai Hospital: Model Fine-Tuning and Validation Study.JMIR formative research · 2026Article
- Article
- Information extraction from clinical notes: are we ready to switch to large language models?Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Temporal Annotation of German Clinical Language in Real and Synthetic Clinical Documents: Corpus Development and Baseline Tagger Validation Study.Journal of medical Internet research · 2026Article
- A Dataset of Psychiatric Hospital Notes with Temporal Information Annotations.LREC ... International Conference on Language Resources & Evaluation : [proceedings]. International Conference on Language Resources & Evaluation · 2026Article
- Forecasting Clinical Risk from Textual Time Series: Structuring Narratives for Temporal AI in Healthcare.medRxiv : the preprint server for health sciences · 2025Article
- HONeYBEE: enabling scalable multimodal AI in oncology through foundation model-driven embeddings.NPJ digital medicine · 2025Article
- Systemic Anticancer Therapy Timelines Extraction From Electronic Medical Records Text: Algorithm Development and Validation.JMIR bioinformatics and biotechnology · 2025Article
- Toward Cross-Hospital Deployment of Natural Language Processing Systems: Model Development and Validation of Fine-Tuned Large Language Models for Disease Name Recognition in Japanese.JMIR medical informatics · 2025Article
- Hierarchical embedding attention for overall survival prediction in lung cancer from unstructured EHRs.BMC medical informatics and decision making · 2025Article
- LLM-IE: a python package for biomedical generative information extraction with large language models.JAMIA open · 2025Article
113 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
Abstract
backgroundThe Sixth Informatics for Integrating Biology and the Bedside (i2b2) Natural Language Processing Challenge for Clinical Records focused on the temporal relations in clinical narratives. The organizers provided the research community with a corpus of discharge summaries annotated with temporal information, to be used for the development and evaluation of temporal reasoning systems. 18 teams from around the world participated in the challenge. During the workshop, participating teams presented comprehensive reviews and analysis of their systems, and outlined future research directions suggested by the challenge contributions.
methodsThe challenge evaluated systems on the information extraction tasks that targeted: (1) clinically significant events, including both clinical concepts such as problems, tests, treatments, and clinical departments, and events relevant to the patient's clinical timeline, such as admissions, transfers between departments, etc; (2) temporal expressions, referring to the dates, times, durations, or frequencies phrases in the clinical text. The values of the extracted temporal expressions had to be normalized to an ISO specification standard; and (3) temporal relations, between the clinical events and temporal expressions. Participants determined pairs of events and temporal expressions that exhibited a temporal relation, and identified the temporal relation between them.
resultsFor event detection, statistical machine learning (ML) methods consistently showed superior performance. While ML and rule based methods seemed to detect temporal expressions equally well, the best systems overwhelmingly adopted a rule based approach for value normalization. For temporal relation classification, the systems using hybrid approaches that combined ML and heuristics based methods produced the best results.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.