Evidence map›Paper›PMID 38997595›Full record

ArticleNature methods2024

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics.

Caleb Weinreb, Jonah E Pearl, Sherry Lin, Mohammed Abdal Monium Osman, Libby Zhang, Sidharth Annapragada, Eli Conlin, Red Hoffmann, Sofia Makowska, Winthrop F Gillis and 7 more

Abstract read
In one paragraph

Article in Nature methods, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 133 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
133citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

133 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. ABEL: an active-learning behavior estimation and labeling platform.bioRxiv : the preprint server for biology · 2026
    Article
  5. Article
  6. Cross-species identification of conserved and divergent locomotor kinematic strategies using AutoGaitA.Proceedings of the National Academy of Sciences of the United States of America · 2026
    Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Review
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Review
  20. Article

73 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

17 authors.

Caleb WeinrebDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.
Jonah E PearlDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.ORCID http://orcid.org/0000-0001-9300-1552
Sherry LinDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.ORCID http://orcid.org/0000-0002-1275-8391
Mohammed Abdal Monium OsmanDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.ORCID http://orcid.org/0000-0001-8606-6518
Libby ZhangDepartment of Electrical Engineering, Stanford University, Stanford, CA, USA.ORCID http://orcid.org/0000-0003-2069-4081
Sidharth AnnapragadaDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.
Eli ConlinDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.
Red HoffmannDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.
Sofia MakowskaDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.
Winthrop F GillisDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.
Maya JayDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA.
Shaokai YeBrain Mind and Neuro-X Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.ORCID http://orcid.org/0000-0003-4250-2220
Alexander MathisBrain Mind and Neuro-X Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.ORCID http://orcid.org/0000-0002-3777-2202
Mackenzie W MathisBrain Mind and Neuro-X Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.ORCID http://orcid.org/0000-0001-7368-4456
Talmo PereiraSalk Institute for Biological Studies, La Jolla, CA, USA.ORCID http://orcid.org/0000-0001-9075-8365
Scott W LindermanWu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA. scott.linderman@stanford.edu.ORCID http://orcid.org/0000-0002-3878-9073
Sandeep Robert DattaDepartment of Neurobiology, Harvard Medical School, Boston, MA, USA. srdatta@hms.harvard.edu.ORCID http://orcid.org/0000-0002-8068-3862

Funding

Viral ModuleP30EY012196 · NEI · HARVARD UNIVERSITY (MEDICAL SCHOOL) · PI ZHIGANG HE · 1998 to 2026
$21.3M
Towards a Unified Framework for Dopamine Signaling in the StriatumU19NS113201 · NINDS · HARVARD MEDICAL SCHOOL · PI SABATINI, BERNARDO L · 2019 to 2023
$18.7M
Motion Sequencing for All: Pipelining, Distribution and Training to Enable Broad Adoption of a Next-Generation Platform for Behavioral and Neurobehavioral AnalysisU24NS109520 · NINDS · HARVARD MEDICAL SCHOOL · PI DATTA, SANDEEP R · 2019 to 2024
$2.3M
CounterAct Administrative Supplement to NS114020 Automated Phenotyping in EpilepsyR01NS114020 · NINDS · STANFORD UNIVERSITY · PI DATTA, SANDEEP R, SOLTESZ, IVAN · 2019 to 2023
$2.3M
Neurobehavioral phenotyping of AD model mice using Motion SequencingRF1AG073625 · NIA · HARVARD MEDICAL SCHOOL · PI DATTA, SANDEEP R · 2021 to 2021
$1.9M
Dissecting the role of the direct and indirect pathways in moment-to-moment action selection.F31NS113385 · NINDS · HARVARD MEDICAL SCHOOL · PI GILLIS, WINTHROP · 2019 to 2021
$106k
Investigating a Role for Dopamine in Organizing Behavioral SequencingF31NS122155 · NINDS · HARVARD MEDICAL SCHOOL · PI JAY, MAYA · 2022 to 2023
$55k
NEI NIH HHS P30 EY012196NIA NIH HHS RF1 AG073625NINDS NIH HHS F31 NS113385NINDS NIH HHS F31 NS122155NINDS NIH HHS R01 NS114020NINDS NIH HHS U19 NS113201NINDS NIH HHS U24 NS109520U.S. Department of Health & Human Services | National Institutes of Health (NIH) F31NS113385U.S. Department of Health & Human Services | National Institutes of Health (NIH) F31NS122155U.S. Department of Health & Human Services | National Institutes of Health (NIH) R01NS114020U.S. Department of Health & Human Services | National Institutes of Health (NIH) RF1AG073625U.S. Department of Health & Human Services | National Institutes of Health (NIH) U24NS109520
6 · The paper itself

Abstract

Keypoint tracking algorithms can flexibly quantify animal movement from videos obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into discrete actions. This challenge is particularly acute because keypoint data are susceptible to high-frequency jitter that clustering algorithms can mistake for transitions between actions. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ('syllables') from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to identify syllables whose boundaries correspond to natural sub-second discontinuities in pose dynamics. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq also works in multiple species and generalizes beyond the syllable timescale, identifying fast sniff-aligned movements in mice and a spectrum of oscillatory behaviors in fruit flies. Keypoint-MoSeq, therefore, renders accessible the modular structure of behavior through standard video recordings.

Indexed as

AlgorithmsBehavior, AnimalMachine LearningVideo RecordingAnimalsDrosophila melanogasterHumansMaleMiceMovement

Identifiers

PMID38997595
PMCPMC11245396

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.