ArticleNature biomedical engineering2023
A vision transformer for decoding surgeon activity from surgical videos.
Article in Nature biomedical engineering, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 70 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
70 citing papers in PubMed, 5 syntheses or guidelines pooled it, 123 citations in OpenAlex.
- Untangling surgical gesture analysis-are we even speaking the same language? a systematic review.Surgical endoscopy · 2025Pooled it
- The Advent of Artificial Intelligence into Cardiac Surgery: A Systematic Review of Our Understanding.Brazilian journal of cardiovascular surgery · 2024Pooled it
- Artificial Intelligence in Cataract Surgery: A Systematic Review.Translational vision science & technology · 2024Pooled it
- Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review.The British journal of surgery · 2024Pooled it
- Technical skill assessment in minimally invasive surgery using artificial intelligence: a systematic review.Surgical endoscopy · 2023Pooled it
- Artificial intelligence-based action recognition and skill assessment in robotic cardiac surgery simulation: a feasibility study.Journal of robotic surgery · 2025Trial
- Artificial Intelligence-Based Video Feedback to Improve Novice Performance on Robotic Suturing Skills: A Pilot Study.Journal of endourology · 2024Trial
- Real-Time multifaceted artificial intelligence vs In-Person instruction in teaching surgical technical skills: a randomized controlled trial.Scientific reports · 2024Trial
- Ambient intelligence-based framework for quantitative evaluation of cell culture operations to support standardization and training.Regenerative therapy · 2026Article
- Decoding surgical competence: surgical gesture analysis can determine competence in simulated VATS lobectomy, a validation study.Surgical endoscopy · 2026Article
- Evaluating Injection Laryngoplasty Skills Using a Foundation Model: A Feasibility Study.The Laryngoscope · 2026Article
- Development of an Artificial Intelligence Platform for Surgical Tool Detection and Phase Recognition in Laparoscopic Distal Gastrectomy.Journal of gastric cancer · 2026Article
- MiDAS: A Multimodal Data Acquisition System and Dataset for Robot-Assisted Minimally Invasive Surgery.The international journal of medical robotics + computer assisted surgery : MRCAS · 2026Article
- Incorporating AI-Driven Vision Systems to Quantify Learning Curve in EVD Placement.Journal of neurological surgery. Part B, Skull base · 2026Article
- Privacy-Preserving Surgical Video Analysis with Swarm Learning - Results from a Multinational Appendectomy Cohort.NEJM AI · 2026Article
- End to end AI system for surgical gesture sequence recognition and clinical outcome prediction.NPJ digital medicine · 2026Article
- Artificial intelligence in urology training Enhancing annotation, feedback, and evaluation in robotic, laparoscopic, and endoscopic surgery.Canadian Urological Association journal = Journal de l'Association des urologues du Canada · 2026Review
- Towards a neuro-symbolic approach for precision anti-reflux surgery.Updates in surgery · 2026Review
- Standardization of surgical gesture taxonomy: a SAGES Delphi consensus study.Surgical endoscopy · 2026Article
- The future of robotic surgery in the age of artificial intelligence.Nature reviews. Urology · 2026Review
10 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors at 4 institutions in 2 countries.
Funding
Abstract
The intraoperative activity of a surgeon has substantial impact on postoperative outcomes. However, for most surgical procedures, the details of intraoperative surgical actions, which can vary widely, are not well understood. Here we report a machine learning system leveraging a vision transformer and supervised contrastive learning for the decoding of elements of intraoperative surgical activity from videos commonly collected during robotic surgeries. The system accurately identified surgical steps, actions performed by the surgeon, the quality of these actions and the relative contribution of individual video frames to the decoding of the actions. Through extensive testing on data from three different hospitals located in two different continents, we show that the system generalizes across videos, surgeons, hospitals and surgical procedures, and that it can provide information on surgical gestures and skills from unannotated videos. Decoding intraoperative activity via accurate machine learning systems could be used to provide surgeons with feedback on their operating skills, and may allow for the identification of optimal surgical behaviour and for the study of relationships between intraoperative factors and postoperative outcomes.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.