Evidence map›Paper›PMID 35394438›Full record

ArticleJMIR pediatrics and parenting2022

Improved Digital Therapy for Developmental Pediatrics Using Domain-Specific Artificial Intelligence: Machine Learning Study.

Peter Washington, Haik Kalantarian, John Kent, Arman Husic, Aaron Kline, Emilie Leblanc, Cathy Hou, Onur Cezmi Mutlu, Kaitlyn Dunlap, Yordan Penev and 9 more

Open access · goldAbstract read
In one paragraph

Article in JMIR pediatrics and parenting, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
17citing papers in PubMed, 1 pooled it
4.5field-weighted citation impact, top 4% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

17 citing papers in PubMed, 1 synthesis or guideline pooled it, 39 citations in OpenAlex.

  1. Pooled it
  2. Review
  3. Article
  4. Article
  5. Article
  6. Review
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Review
  17. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

19 authors at 1 institution in 1 country.

Peter WashingtonDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0003-3276-4411
Haik KalantarianDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-7107-7908
John KentDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-7989-6596
Arman HusicDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-9180-5212
Aaron KlineDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-0077-5485
Emilie LeblancDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-3492-3554
Cathy HouDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0001-6766-5128
Onur Cezmi MutluDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-9263-9332
Kaitlyn DunlapDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0003-4423-5269
Yordan PenevDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0001-8520-9417
Maya VarmaDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0003-0693-7753
Nate Tyler StockhamDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-0752-6801
Brianna ChrismanDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-7157-607X
Kelley PaskovDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-5252-1401
Min Woo SunDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0003-1049-1854
Jae-Yoon JungDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0001-7948-9803
Catalin VossDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0001-6480-7020
Nick HaberDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0001-8804-7804
Dennis Paul WallDepartments of Pediatrics (Systems Medicine) and Biomedical Data Science, Stanford University, Stanford, CA, United States.ORCID https://orcid.org/0000-0002-7889-9146
Stanford University · US

Funding

A Mobile Game for Domain Adaptation and Deep Learning in Autism HealthcareR01LM013364 · NLM · STANFORD UNIVERSITY · PI WALL, DENNIS PAUL · 2021 to 2025
$3.2M
Creating an artificial intelligence therapy-to-data feedback loop for child developmental healthcareR01LM013083 · NLM · STANFORD UNIVERSITY · PI WALL, DENNIS PAUL · 2019 to 2022
$2.6M
QuBBD: Wearable artificial intelligence for bid data-driven healthcare in child development R01EB025025 · NIBIB · STANFORD UNIVERSITY · PI WALL, DENNIS PAUL · 2017 to 2019
$1.1M
Evaluation of machine learning to mobilize detection and therapy of developmental delay in childrenR21HD091500 · NICHD · STANFORD UNIVERSITY · PI WALL, DENNIS PAUL · 2017 to 2018
$432k
NIBIB NIH HHS R01 EB025025NICHD NIH HHS R21 HD091500NLM NIH HHS R01 LM013083NLM NIH HHS R01 LM013364
6 · The paper itself

Abstract

backgroundAutomated emotion classification could aid those who struggle to recognize emotions, including children with developmental behavioral conditions such as autism. However, most computer vision emotion recognition models are trained on adult emotion and therefore underperform when applied to child faces.

objectiveWe designed a strategy to gamify the collection and labeling of child emotion-enriched images to boost the performance of automatic child emotion recognition models to a level closer to what will be needed for digital health care approaches.

methodsWe leveraged our prototype therapeutic smartphone game, GuessWhat, which was designed in large part for children with developmental and behavioral conditions, to gamify the secure collection of video data of children expressing a variety of emotions prompted by the game. Independently, we created a secure web interface to gamify the human labeling effort, called HollywoodSquares, tailored for use by any qualified labeler. We gathered and labeled 2155 videos, 39,968 emotion frames, and 106,001 labels on all images. With this drastically expanded pediatric emotion-centric database (>30 times larger than existing public pediatric emotion data sets), we trained a convolutional neural network (CNN) computer vision classifier of happy, sad, surprised, fearful, angry, disgust, and neutral expressions evoked by children.

resultsThe classifier achieved a 66.9% balanced accuracy and 67.4% F1-score on the entirety of the Child Affective Facial Expression (CAFE) as well as a 79.1% balanced accuracy and 78% F1-score on CAFE Subset A, a subset containing at least 60% human agreement on emotions labels. This performance is at least 10% higher than all previously developed classifiers evaluated against CAFE, the best of which reached a 56% balanced accuracy even when combining "anger" and "disgust" into a single class.

conclusionsThis work validates that mobile games designed for pediatric therapies can generate high volumes of domain-relevant data sets to train state-of-the-art classifiers to perform tasks helpful to precision health efforts.

Indexed as

affective computingartificial intelligenceautism spectrum disordercomputer visionconvolutional neural networkdigital therapyemotion recognitionmachine learningmobile healthpediatrics

Identifiers

PMID35394438
PMCPMC9034430
OpenAlexW4205785225

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.