ArticleIEEE transactions on medical imaging2024
Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting From Multimodal Data.
Article in IEEE transactions on medical imaging, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 15 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
15 citing papers in PubMed.
- A Systematic Review of Deep Learning and Machine Learning Applications in Longitudinal Multimodal Clinical Data.Journal of healthcare informatics research · 2026Review
- From Integrated Care to Learning Systems.Healthcare (Basel, Switzerland) · 2026Review
- An External Validation Study on Two Pre-Trained Large Language Models for Multimodal Prognostication in Laryngeal and Hypopharyngeal Cancer: Integrating Clinical, Treatment, and Radiomic Data to Predict Survival Outcomes with Interpretable Reasoning.Bioengineering (Basel, Switzerland) · 2025Article
- Plagued by the Past, Pressed by the Present: A One Health Perspective onBiomedicines · 2025Review
- The future of multimodal artificial intelligence models for integrating imaging and clinical metadata: a narrative reviewDiagnostic and interventional radiology (Ankara, Turkey) · 2025Review
- An intelligent diagnostic method for porcine gastrointestinal infectious diseases based on multimodal AI and large language model.Frontiers in veterinary science · 2025Article
- Application of artificial intelligence in Alzheimer's disease: a bibliometric analysis.Frontiers in neuroscience · 2025Review
- Multimodality Fusion Aspects of Medical Diagnosis: A Comprehensive Review.Bioengineering (Basel, Switzerland) · 2024Review
- Big Epidemiology: The Birth, Life, Death, and Resurgence of Diseases on a Global Timescale.Epidemiologia (Basel, Switzerland) · 2024Review
- Artificial intelligence enhances the management of esophageal squamous cell carcinoma in the precision oncology era.World journal of gastroenterology · 2024Review
- Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting From Multimodal Data.IEEE transactions on medical imaging · 2024Article
- Hybrid Fusion of High-Resolution and Ultra-Widefield OCTA Acquisitions for the Automatic Diagnosis of Diabetic Retinopathy.Diagnostics (Basel, Switzerland) · 2023Article
- Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data.Scientific reports · 2023Article
- The KNee OsteoArthritis Prediction (KNOAP2020) challenge: An image analysis challenge to predict incident symptomatic radiographic knee osteoarthritis from MRI and X-ray images.Osteoarthritis and cartilage · 2023Article
- Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
Abstract
Deep neural networks are often applied to medical images to automate the problem of medical diagnosis. However, a more clinically relevant question that practitioners usually face is how to predict the future trajectory of a disease. Current methods for prognosis or disease trajectory forecasting often require domain knowledge and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many prediction problem. Inspired by a clinical decision-making process with two agents-a radiologist and a general practitioner - we predict prognosis with two transformer-based components that share information with each other. The first transformer in this framework aims to analyze the imaging data, and the second one leverages its internal states as inputs, also fusing them with auxiliary clinical data. The temporal nature of the problem is modeled within the transformer states, allowing us to treat the forecasting problem as a multi-task classification, for which we propose a novel loss. We show the effectiveness of our approach in predicting the development of structural knee osteoarthritis changes and forecasting Alzheimer's disease clinical status directly from raw multi-modal data. The proposed method outperforms multiple state-of-the-art baselines with respect to performance and calibration, both of which are needed for real-world applications. An open-source implementation of our method is made publicly available at https://github.com/Oulu-IMEDS/CLIMATv2.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.