SynthesisActa orthopaedica2021
Availability and reporting quality of external validations of machine-learning prediction models with orthopedic surgical outcomes: a systematic review.
Synthesis in Acta orthopaedica, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 36 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
36 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Performance of prediction models for delayed union and nonunion after fracture: a systematic review and meta-analysis.BMC musculoskeletal disorders · 2026Pooled it
- TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods: a Korean translation.Ewha medical journal · 2025Guideline
- Clinical prediction models for patients undergoing total hip arthroplasty: an external validation based on a systematic review and the Dutch Arthroplasty Register.Acta orthopaedica · 2024Pooled it
- Machine learning prediction models in orthopedic surgery: A systematic review in transparent reporting.Journal of orthopaedic research : official publication of the Orthopaedic Research Society · 2022Pooled it
- Advanced Machine Learning Models for Predicting Post-Thrombolysis Hemorrhagic Transformation in Acute Ischemic Stroke Patients: A Systematic Review and Meta-Analysis.Clinical and applied thrombosis/hemostasis : official journal of the International Academy of Clinical and Applied Thrombosis/HemostasisPooled it
- Artificial Intelligence in Arthroplasty: A Comprehensive Structured Critical Review and Descriptive Evidence Map of Validation, Uncertainty, and Workflow Integration.Bioengineering (Basel, Switzerland) · 2026Review
- Artificial intelligence and machine learning: an important new set of tools for clinical shoulder arthroplasty research.JSES international · 2026Review
- Re: Methodological Considerations in Frailty Assessment for Distal Radius Fracture Surgery.Geriatric orthopaedic surgery & rehabilitation · 2026Article
- Artificial intelligence in orthopedic regenerative medicine: from design to clinical translational pathways.Frontiers in cell and developmental biology · 2026Review
- The maximum 2D diameter column of the notch as the most important bone risk indicator compared with the posterior tibial slopes for ACL injury based on computed tomography: Analysis using machine learning approach.Journal of experimental orthopaedics · 2026Article
- Interpretable Multiomics Models for Predicting Surgical Interventions and Blood Transfusion Requirements in Traumatic Brain Injury.NPJ digital medicine · 2025Article
- Secondary Prevention of AFAIS: Deploying Traditional Regression, Machine Learning, and Deep Learning Models to Validate and Update CHA2DS2-VASc for 90-Day Recurrence.Journal of clinical medicine · 2025Article
- From data to precision: The transformative role of AI and machine learning in modern orthopaedic practice.Journal of clinical orthopaedics and trauma · 2025Article
- Temporal validation of the SORG 90-Day and 1-Year machine learning algorithms for survival of patients with spinal metastatic disease.European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society · 2025Article
- PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods.BMJ (Clinical research ed.) · 2025Article
- Prediction Tools in Spine Surgery: A Narrative Review.Spine surgery and related research · 2025Review
- Revolutionizing total hip arthroplasty: The role of artificial intelligence and machine learning.Journal of experimental orthopaedics · 2025Article
- powerROC: An Interactive Web Tool for Sample Size Calculation in Assessing Models' Discriminative Abilities.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
- Non-invasive prediction of human embryonic ploidy using artificial intelligence: a systematic review and meta-analysis.EClinicalMedicine · 2024Article
- External validation of an artificial intelligence multi-label deep learning model capable of ankle fracture classification.BMC musculoskeletal disorders · 2024Observational
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
12 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background and purpose - External validation of machine learning (ML) prediction models is an essential step before clinical application. We assessed the proportion, performance, and transparent reporting of externally validated ML prediction models in orthopedic surgery, using the Transparent Reporting for Individual Prognosis or Diagnosis (TRIPOD) guidelines.Material and methods - We performed a systematic search using synonyms for every orthopedic specialty, ML, and external validation. The proportion was determined by using 59 ML prediction models with only internal validation in orthopedic surgical outcome published up until June 18, 2020, previously identified by our group. Model performance was evaluated using discrimination, calibration, and decision-curve analysis. The TRIPOD guidelines assessed transparent reporting.Results - We included 18 studies externally validating 10 different ML prediction models of the 59 available ML models after screening 4,682 studies. All external validations identified in this review retained good discrimination. Other key performance measures were provided in only 3 studies, rendering overall performance evaluation difficult. The overall median TRIPOD completeness was 61% (IQR 43-89), with 6 items being reported in less than 4/18 of the studies.Interpretation - Most current predictive ML models are not externally validated. The 18 available external validation studies were characterized by incomplete reporting of performance measures, limiting a transparent examination of model performance. Further prospective studies are needed to validate or refute the myriad of predictive ML models in orthopedics while adhering to existing guidelines. This ensures clinicians can take full advantage of validated and clinically implementable ML decision tools.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.