Evidence map›Paper›PMID 39951536›Full record

ArticleScience advances2025

Artificial intelligence-powered 3D analysis of video-based caregiver-child interactions.

Zhenzhen Weng, Laura Bravo-Sánchez, Zeyu Wang, Christopher Howard, Maria Xenochristou, Nicole Meister, Angjoo Kanazawa, Arnold Milstein, Elika Bergelson, Kathryn L Humphreys and 2 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Science advances, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 3 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
3citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

3 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Deep learning in fetal, infant, and toddler neuroimaging research.Developmental cognitive neuroscience · 2026
    Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

12 authors.

Zhenzhen WengInstitute for Computational & Mathematical Engineering, Stanford University, Stanford, CA, USA.ORCID 0009-0004-1108-4155
Laura Bravo-SánchezDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.ORCID 0000-0003-3556-3391
Zeyu WangDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.
Christopher HowardDepartment of Computer Science, Stanford University, Stanford, CA, USA.
Maria XenochristouDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.
Nicole MeisterDepartment of Electrical Engineering, Stanford University, Stanford, CA, USA.ORCID 0000-0002-7154-6882
Angjoo KanazawaDepartment of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA, USA.ORCID 0000-0003-2592-8430
Arnold MilsteinClinical Excellence Research Center, Stanford University, Stanford, CA, USA.ORCID 0000-0002-2794-2378
Elika BergelsonDepartment of Psychology, Harvard University, Cambridge, MA, USA.ORCID 0000-0003-2742-4797
Kathryn L HumphreysDepartment of Psychology and Human Development, Vanderbilt University, Nashville, TN, USA.ORCID 0000-0002-5715-6597
Lee M SandersPediatrics and Health Policy, Stanford University, Stanford, CA, USA.ORCID 0000-0003-3231-6918
Serena Yeung-LevyDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.ORCID 0000-0003-0529-0628

Funding

Parent-child proximity and emerging psychopathologyR01MH129634 · NIMH · VANDERBILT UNIVERSITY · PI Kathryn Leigh Humphreys · 2022 to 2026
$3.9M
Research Supplement for Immigrant Families and Children's HealthR01MD013844 · NIMHD · STANFORD UNIVERSITY · PI HAINMUELLER, JENS · 2020 to 2023
$2.5M
Mechanisms of Word Learning in InfancyDP5OD019812 · OD · UNIVERSITY OF ROCHESTER · PI BERGELSON, ELIKA · 2014 to 2018
$1.9M
NIH HHS DP5 OD019812NIMHD NIH HHS R01 MD013844NIMH NIH HHS R01 MH129634
6 · The paper itself

Abstract

We introduce HARMONI, a three-dimensional (3D) computer vision and audio processing method for analyzing caregiver-child behavior and interaction from observational videos. HARMONI operates at subsecond resolution, estimating 3D mesh representations and spatial interactions of humans, and adapts to challenging natural environments using an environment-targeted synthetic data generation module. Deployed on 500 hours from the SEEDLingS dataset, HARMONI generates detailed quantitative measurements of 3D human behavior previously unattainable through manual efforts or 2D methods. HARMONI identifies longitudinal trends in child-caregiver interaction, including child movement, body pose, dyadic touch, visibility, and conversational turns. The integrated visual and audio analysis further reveals multimodal trends, including associations between child conversational turns and movement. Open-sourced for large-scale analysis, HARMONI facilitates advancements in human development research. HARMONI achieves 63 to 80% consistency on key attributes with human annotators on SEEDLingS and 84 to 93% consistency on videos taken from a laboratory setting while achieving >100 times savings in time.

Indexed as

Artificial IntelligenceCaregiversImaging, Three-DimensionalVideo RecordingChildFemaleHumansMale

Identifiers

PMID39951536
PMCPMC11837989

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.