Evidence map›Paper›PMID 34600518›Full record

ArticleBMC medical informatics and decision making2021

A framework for validating AI in precision medicine: considerations from the European ITFoC consortium.

Rosy Tsopra, Xose Fernandez, Claudio Luchinat, Lilia Alberghina, Hans Lehrach, Marco Vanoni, Felix Dreher, O Ugur Sezerman, Marc Cuggia, Marie de Tayrac and 24 more

Open access · goldAbstract read
In one paragraph

Article in BMC medical informatics and decision making, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 46 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
46citing papers in PubMed, 2 pooled it
8.6field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

46 citing papers in PubMed, 2 syntheses or guidelines pooled it, 95 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Review
  4. Article
  5. Review
  6. Article
  7. Article
  8. New Personalized Medicine Model for Medication Management.Journal of personalized medicine · 2026
    Review
  9. Article
  10. Article
  11. Review
  12. Article
  13. Review
  14. Review
  15. Review
  16. Article
  17. Review
  18. Real-World Data and Causal Machine Learning to Enhance Drug Development.Therapeutic innovation & regulatory science · 2025
    Review
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

34 authors at 14 institutions in 6 countries.

Rosy TsopraCentre de Recherche Des Cordeliers, Inserm, Université de Paris, Sorbonne Université, 75006, Paris, France. rosy.tsopra@nhs.net.ORCID 0000-0002-9406-5547
Xose FernandezInstitut Curie, 25 Rue d'Ulm, 75005, Paris, France.
Claudio LuchinatCentro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy.
Lilia AlberghinaDepartment of Biotechnology and Biosciences, University of Milano Bicocca and ISBE-Italy/SYSBIO - Candidate National Node of Italy for ISBE, Research Infrastructure for Systems Biology Europe, Milan, Italy.
Hans LehrachMax Planck Institute for Molecular Genetics, Berlin, Germany.
Marco VanoniDepartment of Biotechnology and Biosciences, University of Milano Bicocca and ISBE-Italy/SYSBIO - Candidate National Node of Italy for ISBE, Research Infrastructure for Systems Biology Europe, Milan, Italy.
Felix DreherAlacris Theranostics GmbH, Berlin, Germany.
O Ugur SezermanSchool of Medicine Biostatistics and Medical Informatics Dept., Acibadem University, Istanbul, Turkey.
Marc CuggiaUniv Rennes, CHU Rennes, Inserm, LTSI - UMR 1099, 35000, Rennes, France.
Marie de TayracUniv Rennes, Department of Molecular Genetics and Genomics, CHU Rennes, IGDR-UMR6290, CNRS, 35000, Rennes, France.
Edvins MiklasevicsRSU Institute of Oncology, Dzirciema str. 16, Riga, 1010, Latvia.
Lucian Mihai ItuTransilvania University of Brasov, Brasov, Romania.
Marius GeantaCentre for Innovation in Medicine, Bucharest, Romania.
Lesley OgilvieMax Planck Institute for Molecular Genetics, Berlin, Germany.
Florence GodeyINSERM U1242 « Chemistry, Oncogenesis Stress Signaling », Université de Rennes, 35042, CEDEX, Rennes, France.
Cristian Nicolae BoldisorTransilvania University of Brasov, Brasov, Romania.
Boris Campillo-GimenezUniv Rennes, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, 35000, Rennes, France.
Cosmina CioroboiuCentre for Innovation in Medicine, Bucharest, Romania.
Costin Florian CiusdelTransilvania University of Brasov, Brasov, Romania.
Simona ComanTransilvania University of Brasov, Brasov, Romania.
Oliver Hijano CubelosInstitut Curie, 25 Rue d'Ulm, 75005, Paris, France.
Alina ItuTransilvania University of Brasov, Brasov, Romania.
Bodo LangeAlacris Theranostics GmbH, Berlin, Germany.
Matthieu Le GalloINSERM U1242 « Chemistry, Oncogenesis Stress Signaling », Université de Rennes, 35042, CEDEX, Rennes, France.
Alexandra LespagnolDepartment of Molecular Genetics and Genomics, CHU Rennes, 35000, Rennes, France.
Giancarlo MauriDepartment of Informatics, Systems and Communication, University of Milano Bicocca and ISBE-Italy/SYSBIO - Candidate National Node of Italy for ISBE, Research Infrastructure for Systems Biology Europe, Milan, Italy.
H Okan SoykamEPIGENETICS Inc. BUDOTEK, Istanbul, Turkey.
Bastien RanceCentre de Recherche Des Cordeliers, Inserm, Université de Paris, Sorbonne Université, 75006, Paris, France.
Paola TuranoCentro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy.
Leonardo TenoriCentro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy.
Alessia VignoliCentro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy.
Christoph WierlingAlacris Theranostics GmbH, Berlin, Germany.
Nora BenhabilesDirection de La Recherche Fondamentale (DRF), CEA, Université Paris-Saclay, 91191, Gif-sur-Yvette, France.
Anita BurgunCentre de Recherche Des Cordeliers, Inserm, Université de Paris, Sorbonne Université, 75006, Paris, France.
Transylvania University of Brașov · ROInserm · FRInteruniversity Consortium for Magnetic Resonance · ITAlacris (Germany) · DEInstitut national de recherche en sciences et technologies du numérique · FRUniversity of Milano-Bicocca · ITInstitut Curie · FRMax Planck Institute for Molecular Genetics · DEMinisterul Cercetării și Inovării · ROAcıbadem University · TRCentre National de la Recherche Scientifique · FRCommissariat à l'Énergie Atomique et aux Énergies Alternatives · FRInstitut de génétique et de développement de Rennes · FRRiga Stradiņš University · LV

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundArtificial intelligence (AI) has the potential to transform our healthcare systems significantly. New AI technologies based on machine learning approaches should play a key role in clinical decision-making in the future. However, their implementation in health care settings remains limited, mostly due to a lack of robust validation procedures. There is a need to develop reliable assessment frameworks for the clinical validation of AI. We present here an approach for assessing AI for predicting treatment response in triple-negative breast cancer (TNBC), using real-world data and molecular -omics data from clinical data warehouses and biobanks.

methodsThe European "ITFoC (Information Technology for the Future Of Cancer)" consortium designed a framework for the clinical validation of AI technologies for predicting treatment response in oncology.

resultsThis framework is based on seven key steps specifying: (1) the intended use of AI, (2) the target population, (3) the timing of AI evaluation, (4) the datasets used for evaluation, (5) the procedures used for ensuring data safety (including data quality, privacy and security), (6) the metrics used for measuring performance, and (7) the procedures used to ensure that the AI is explainable. This framework forms the basis of a validation platform that we are building for the "ITFoC Challenge". This community-wide competition will make it possible to assess and compare AI algorithms for predicting the response to TNBC treatments with external real-world datasets.

conclusionsThe predictive performance and safety of AI technologies must be assessed in a robust, unbiased and transparent manner before their implementation in healthcare settings. We believe that the consideration of the ITFoC consortium will contribute to the safe transfer and implementation of AI in clinical settings, in the context of precision oncology and personalized care.

Indexed as

Artificial IntelligenceNeoplasmsAlgorithmsHumansMachine LearningPrecision MedicineArtificial intelligenceCancerComputerized decision support systemsOncologyPersonalized medicinePrecision medicine

Identifiers

PMID34600518
PMCPMC8487519
OpenAlexW3201841192

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.