Evidence map›Paper›PMID 38918605›Full record

ArticleNature methods2024

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools.

Dan Biderman, Matthew R Whiteway, Cole Hurwitz, Nicholas Greenspan, Robert S Lee, Ankit Vishnubhotla, Richard Warren, Federico Pedraja, Dillon Noone, Michael M Schartner and 11 more

Abstract read
In one paragraph

Article in Nature methods, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 45 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
45citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

45 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

21 authors.

Dan Biderman *Columbia University, New York, NY, USA. db3236@cumc.columbia.edu.ORCID http://orcid.org/0000-0003-2054-8439
Matthew R Whiteway *Columbia University, New York, NY, USA. m.whiteway@columbia.edu.ORCID http://orcid.org/0000-0003-3756-1349
Cole HurwitzColumbia University, New York, NY, USA.
Nicholas GreenspanColumbia University, New York, NY, USA.
Robert S LeeLightning.ai, New York, NY, USA.
Ankit VishnubhotlaColumbia University, New York, NY, USA.
Richard WarrenColumbia University, New York, NY, USA.
Federico PedrajaColumbia University, New York, NY, USA.ORCID http://orcid.org/0000-0002-7056-1159
Dillon NooneColumbia University, New York, NY, USA.
Michael M SchartnerChampalimaud Centre for the Unknown, Lisbon, Portugal.ORCID http://orcid.org/0000-0002-4854-3034
Julia M HuntenburgMax Planck Institute for Biological Cybernetics, Tübingen, Germany.
Anup KhanalUniversity of California, Los Angeles, Los Angeles, CA, USA.
Guido T MeijerChampalimaud Centre for the Unknown, Lisbon, Portugal.
Jean-Paul NoelNew York University, New York, NY, USA.
Alejandro Pan-VazquezPrinceton University, Princeton, NJ, USA.
Karolina Z SochaUniversity College London, London, UK.
Anne E UraiLeiden University, Leiden, the Netherlands.ORCID http://orcid.org/0000-0001-5270-6513
International Brain Laboratory
John P CunninghamColumbia University, New York, NY, USA.ORCID http://orcid.org/0000-0001-9125-3027
Nathaniel B SawtellColumbia University, New York, NY, USA.
Liam PaninskiColumbia University, New York, NY, USA.

Funding

State-dependent Decision-making in Brainwide Neural CircuitsU19NS123716 · NINDS · COLUMBIA UNIV NEW YORK MORNINGSIDE · PI ZADOR, ANTHONY M · 2021 to 2025
$18.2M
Validating Theoretical Models with Neurophysiology and OptogeneticsU19NS107613 · NINDS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI SCANZIANI, MASSIMO · 2018 to 2022
$16.7M
Roles for Granule Cells in Adaptive Processing in a Cerebellum-like CircuitR01NS075023 · NINDS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI ABBOTT, LAURENCE F., SAWTELL, NATHANIEL · 2012 to 2025
$5.8M
Neuroanatomic and Functional Characterization of Cerebellar Circuits Mediating Ingestive BehaviorsR01DK131086 · NIDDK · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI RUDOLPH L LEIBEL, Nathaniel Sawtell · 2022 to 2026
$3.1M
Understanding Multi-Layer Learning in a Biological CircuitRF1NS118448 · NINDS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI ABBOTT, LAURENCE F., SAWTELL, NATHANIEL · 2020 to 2020
$1.5M
Understanding Multi-Layer Learning in a Biological CircuitR01NS118448 · NINDS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI ABBOTT, LAURENCE F., SAWTELL, NATHANIEL · 2023 to 2024
$865k
Neural and Computational Architecture for Complex Navigation and Subjective Self-LocationK99NS128075 · NINDS · NEW YORK UNIVERSITY · PI NOEL, JEAN-PAUL · 2023 to 2024
$253k
Gatsby Charitable Foundation GAT3708National Science Foundation (NSF) 1707398National Science Foundation (NSF) IOS-2115007Nederlandse Organisatie voor Wetenschappelijk Onderzoek (Netherlands Organisation for Scientific Research) VI.Veni.212.184NIDDK NIH HHS R01 DK131086NINDS NIH HHS K99 NS128075NINDS NIH HHS R01 NS075023NINDS NIH HHS R01 NS118448NINDS NIH HHS RF1 NS118448NINDS NIH HHS U19 NS107613NINDS NIH HHS U19 NS123716Simons Foundation 543023U.S. Department of Health & Human Services | NIH | National Institute of Diabetes and Digestive and Kidney Diseases (National Institute of Diabetes & Digestive & Kidney Diseases) 5R01DK131086-02U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (NINDS) 1RF1NS118448-01U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (NINDS) K99NS128075U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (NINDS) NS075023U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (NINDS) U19NS123716Wellcome TrustWellcome Trust 216324
6 · The paper itself

Abstract

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce 'Lightning Pose', an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We released a cloud application that allows users to label data, train networks and process new videos directly from the browser.

Indexed as

AlgorithmsBayes TheoremVideo RecordingAnimalsBehavior, AnimalCloud ComputingDeep LearningImage Processing, Computer-AssistedPostureSoftwareSupervised Machine Learning

Identifiers

PMID38918605
PMCPMC12087009

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.