ReviewNPJ digital medicine2024
The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review.
Review in NPJ digital medicine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 62 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
62 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Deep learning approaches for resolving genomic discrepancies in cancer: a systematic review and clinical perspective.Briefings in bioinformatics · 2025Pooled it
- AI in Medical Questionnaires: Scoping ReviewJournal of medical Internet research · 2025Pooled it
- Research progress of MRI-based radiomics in hepatocellular carcinoma.Frontiers in oncology · 2025Pooled it
- A systematic review of ethical considerations of large language models in healthcare and medicine.Frontiers in digital health · 2025Pooled it
- Revolutionizing diagnosis of pulmonaryFrontiers in microbiology · 2024Pooled it
- An explainable biomedical foundation model via large-scale concept-enhanced vision-language pretraining.Nature biomedical engineering · 2026Article
- Algorithmic Prognostication in Female Oncofertility Counseling: Ethical Challenges of Bias, Autonomy, and Predictive Uncertainty.Healthcare (Basel, Switzerland) · 2026Review
- AI-based augmentation of oncology clinical trials.Nature reviews. Clinical oncology · 2026Review
- RCAF for patient-level thyroid ultrasound malignancy prediction under leakage-free evaluation and calibration.Scientific reports · 2026Article
- Health Data Quality Skill Gaps and Training Needs Among European Health Data Stakeholders: Cross-Sectional Survey.Journal of medical Internet research · 2026Article
- Clinical registry metadata as a hidden bottleneck in AI-driven drug discovery: a computational audit of translational phase data in glioma research.Scientific reports · 2026Article
- Respiratory Disease Detection: A Systematic Review of AI-Based Approaches, from Audio and Visual Unimodal Methods to Multimodal Integration.Diagnostics (Basel, Switzerland) · 2026Review
- A Practical Approach to Assessing the Completeness of Electronic Health Records for Medical Research: Data Quality Study.JMIR medical informatics · 2026Article
- Evaluation of AI tool assisting primary healthcare physicians to diagnostic and treatment tasks.Family medicine and community health · 2026Article
- Importance of integrating biological sex and age analyses in health research.Biology of sex differences · 2026Article
- Current state and potential of hospitals for automated healthcare-associated infection surveillance: data from 24 European countries, 2022 to 2023.Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin · 2026Article
- Challenges in early detection and prognostication of sepsis: new approaches from the emergency department and intensive care unit.EClinicalMedicine · 2026Review
- Radiomic analysis of medical imaging for classification in oncology and recommendations for clinically initiated research: a literature review.Quantitative imaging in medicine and surgery · 2026Review
- Interpretable LightGBM model with SHAP analysis predicts non-excellent response to initial radioiodine therapy in differentiated thyroid carcinoma.Journal of applied clinical medical physics · 2026Article
- AP-Lab: An AI-Driven Autonomous Pilot-Scale Platform Bridging Materials Discovery and Industrial Manufacturing.Advanced science (Weinheim, Baden-Wurttemberg, Germany) · 2026Article
2 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
The adoption of machine learning (ML) and, more specifically, deep learning (DL) applications into all major areas of our lives is underway. The development of trustworthy AI is especially important in medicine due to the large implications for patients' lives. While trustworthiness concerns various aspects including ethical, transparency and safety requirements, we focus on the importance of data quality (training/test) in DL. Since data quality dictates the behaviour of ML products, evaluating data quality will play a key part in the regulatory approval of medical ML products. We perform a systematic review following PRISMA guidelines using the databases Web of Science, PubMed and ACM Digital Library. We identify 5408 studies, out of which 120 records fulfil our eligibility criteria. From this literature, we synthesise the existing knowledge on data quality frameworks and combine it with the perspective of ML applications in medicine. As a result, we propose the METRIC-framework, a specialised data quality framework for medical training data comprising 15 awareness dimensions, along which developers of medical ML applications should investigate the content of a dataset. This knowledge helps to reduce biases as a major source of unfairness, increase robustness, facilitate interpretability and thus lays the foundation for trustworthy AI in medicine. The METRIC-framework may serve as a base for systematically assessing training datasets, establishing reference datasets, and designing test datasets which has the potential to accelerate the approval of medical ML products.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.