Evidence map›Paper›PMID 33734466›Full record

SynthesisJournal of orthopaedic research : official publication of the Orthopaedic Research Society2022

Machine learning prediction models in orthopedic surgery: A systematic review in transparent reporting.

Olivier Q Groot, Paul T Ogink, Amanda Lans, Peter K Twining, Neal D Kapoor, William DiGiovanni, Bas J J Bindels, Michiel E R Bongers, Jacobien H F Oosterhoff, Aditya V Karhade and 3 more

Abstract readSystematic Review
In one paragraph

Synthesis in Journal of orthopaedic research : official publication of the Orthopaedic Research Society, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 41 papers, 5 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
41citing papers in PubMed, 5 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

41 citing papers in PubMed, 5 syntheses or guidelines pooled it.

  1. Guideline
  2. Pooled it
  3. Pooled it
  4. Machine learning prediction models in orthopedic surgery: A systematic review in transparent reporting.Journal of orthopaedic research : official publication of the Orthopaedic Research Society · 2022
    Pooled it
  5. Pooled it
  6. Article
  7. Article
  8. Article
  9. Review
  10. Review
  11. Article
  12. Article
  13. Article
  14. Review
  15. Article
  16. Article
  17. Review
  18. Temporal validation of the SORG 90-Day and 1-Year machine learning algorithms for survival of patients with spinal metastatic disease.European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society · 2025
    Article
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

13 authors.

Olivier Q GrootOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.ORCID http://orcid.org/0000-0002-5091-3110
Paul T OginkDepartment of Orthopedic Surgery, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
Amanda LansOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.
Peter K TwiningOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.ORCID http://orcid.org/0000-0002-7962-7038
Neal D KapoorOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.
William DiGiovanniOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.
Bas J J BindelsDepartment of Orthopedic Surgery, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
Michiel E R BongersOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.
Jacobien H F OosterhoffOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.
Aditya V KarhadeOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.
F C OnerDepartment of Orthopedic Surgery, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
Jorrit-Jan VerlaanDepartment of Orthopedic Surgery, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
Joseph H SchwabOrthopedic Oncology Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Machine learning (ML) studies are becoming increasingly popular in orthopedics but lack a critically appraisal of their adherence to peer-reviewed guidelines. The objective of this review was to (1) evaluate quality and transparent reporting of ML prediction models in orthopedic surgery based on the transparent reporting of multivariable prediction models for individual prognosis or diagnosis (TRIPOD), and (2) assess risk of bias with the Prediction model Risk Of Bias ASsessment Tool. A systematic review was performed to identify all ML prediction studies published in orthopedic surgery through June 18th, 2020. After screening 7138 studies, 59 studies met the study criteria and were included. Two reviewers independently extracted data and discrepancies were resolved by discussion with at least two additional reviewers present. Across all studies, the overall median completeness for the TRIPOD checklist was 53% (interquartile range 47%-60%). The overall risk of bias was low in 44% (n = 26), high in 41% (n = 24), and unclear in 15% (n = 9). High overall risk of bias was driven by incomplete reporting of performance measures, inadequate handling of missing data, and use of small datasets with inadequate outcome numbers. Although the number of ML studies in orthopedic surgery is increasing rapidly, over 40% of the existing models are at high risk of bias. Furthermore, over half incompletely reported their methods and/or performance measures. Until these issues are adequately addressed to give patients and providers trust in ML models, a considerable gap remains between the development of ML prediction models and their implementation in orthopedic practice.

Indexed as

Orthopedic ProceduresOrthopedicsBiasHumansMachine LearningPrognosismachine learningorthopedicsprediction models

Identifiers

PMID33734466
PMCPMC9290012

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.