Evidence map›Paper›PMID 31797581›Full record

Trial reportAnnals of clinical and translational neurology2019

Reproducibility of cognitive endpoints in clinical trials: lessons from neurofibromatosis type 1.

Jonathan M Payne, Stephen J C Hearps, Karin S Walsh, Iris Paltin, Belinda Barton, Nicole J Ullrich, Kristina M Haebich, David Coghill, Gerard A Gioia, Alan Cantor and 15 more

Open access · goldAbstract readClinical Trial, Phase IIMulticenter StudyRandomized Controlled Trial
In one paragraph

Trial report in Annals of clinical and translational neurology, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers, 3 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
23citing papers in PubMed, 3 pooled it
1.9field-weighted citation impact, top 15% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

23 citing papers in PubMed, 3 syntheses or guidelines pooled it, 35 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Trial
  5. Article
  6. Article
  7. Review
  8. Article
  9. Review
  10. Article
  11. Article
  12. Review
  13. Review
  14. Review
  15. Article
  16. Review
  17. Review
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

25 authors at 13 institutions in 2 countries.

Jonathan M PayneMurdoch Children's Research Institute, Royal Children's Hospital, Parkville, Victoria, Australia.ORCID 0000-0001-9565-3845
Stephen J C HearpsMurdoch Children's Research Institute, Royal Children's Hospital, Parkville, Victoria, Australia.
Karin S WalshCenter for Neuroscience and Behavioral Medicine, Children's National Health System, Washington, DC.
Iris PaltinDivision of Oncology, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.
Belinda BartonKids Neuroscience Centre, The Children's Hospital at Westmead, Westmead, New South Wales, Australia.
Nicole J UllrichDepartment of Neurology, Boston Children's Hospital, Boston, Massachusetts.ORCID 0000-0001-5623-3362
Kristina M HaebichMurdoch Children's Research Institute, Royal Children's Hospital, Parkville, Victoria, Australia.
David CoghillMurdoch Children's Research Institute, Royal Children's Hospital, Parkville, Victoria, Australia.
Gerard A GioiaCenter for Neuroscience and Behavioral Medicine, Children's National Health System, Washington, DC.
Alan CantorDepartment of Preventative Medicine, University of Alabama at Birmingham, Birmingham, Alabama.
Gary CutterSchool of Public Health, University of Alabama at Birmingham, Birmingham, Alabama.
James H TonsgardDivision of Neurology, The University of Chicago Medicine Comer Children's Hospital, Chicago, Illinois.
David ViskochilDepartment of Genetics, University of Utah, Salt Lake City, Utah.
Celiane Rey-CasserlyCenter for Neuropsychology, Boston Children's Hospital, Boston, Massachusetts.
Elizabeth K SchorryHuman Genetics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio.
Joseph D AckersonDepartment of Psychology, University of Alabama at Birmingham, Birmingham, Alabama.
Laura KlesseDepartment of Pediatrics, University of Texas Southwestern Medical Center, Dallas, Texas.
Michael J FisherDivision of Oncology, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.
David H GutmannDepartment of Neurology, Washington University School of Medicine, St Louis, Missouri.ORCID 0000-0002-3127-5045
Tena RosserDepartment of Neurology, Children's Hospital of Los Angeles, Los Angeles, California.
Roger J PackerCenter for Neuroscience and Behavioral Medicine, Children's National Health System, Washington, DC.
Bruce KorfDepartment of Genetics, University of Alabama at Birmingham, Birmingham, Alabama.
Maria T AcostaCenter for Neuroscience and Behavioral Medicine, Children's National Health System, Washington, DC.
Kathryn N NorthMurdoch Children's Research Institute, Royal Children's Hospital, Parkville, Victoria, Australia.
NF Clinical Trials Consortium
Royal Children's Hospital · AUUniversity of Alabama at Birmingham · USChildren's National · USBoston Children's Hospital · USChildren's Hospital of Philadelphia · USChildren's Hospital of Los Angeles · USCincinnati Children's Hospital Medical Center · USComer Children's Hospital · USNational Institutes of Health · USThe University of Sydney · AUThe University of Texas Southwestern Medical Center · USUniversity of Utah · USWashington University in St. Louis · US

Funding

Research_ProjectU54HD090257 · NICHD · CHILDREN'S RESEARCH INSTITUTE · PI VITTORIO GALLO, VITTORIO GALLO VITTORIO GALLO · 2016 to 2020
$5.9M
NICHD NIH HHS U54 HD090257US Department of Defense W81XWH-05-1-0615
6 · The paper itself

Abstract

objectiveRapid developments in understanding the molecular mechanisms underlying cognitive deficits in neurodevelopmental disorders have increased expectations for targeted, mechanism-based treatments. However, translation from preclinical models to human clinical trials has proven challenging. Poor reproducibility of cognitive endpoints may provide one explanation for this finding. We examined the suitability of cognitive outcomes for clinical trials in children with neurofibromatosis type 1 (NF1) by examining test-retest reliability of the measures and the application of data reduction techniques to improve reproducibility.

methodsData were analyzed from the STARS clinical trial (n = 146), a multi-center double-blind placebo-controlled phase II trial of lovastatin, conducted by the NF Clinical Trials Consortium. Intra-class correlation coefficients were generated between pre- and post-performances (16-week interval) on neuropsychological endpoints in the placebo group to determine test-retest reliabilities. Confirmatory factor analysis was used to reduce data into cognitive domains and account for measurement error.

resultsTest-retest reliabilities were highly variable, with most endpoints demonstrating unacceptably low reproducibility. Data reduction confirmed four distinct neuropsychological domains: executive functioning/attention, visuospatial ability, memory, and behavior. Test-retest reliabilities of latent factors improved to acceptable levels for clinical trials. Applicability and utility of our model was demonstrated by homogeneous effect sizes in the reanalyzed efficacy data.

interpretationThese data demonstrate that single observed endpoints are not appropriate to determine efficacy, partly accounting for the poor test-retest reliability of cognitive outcomes in clinical trials in neurodevelopmental disorders. Recommendations to improve reproducibility are outlined to guide future trial design.

Indexed as

Neurofibromatosis 1Reproducibility of ResultsAdolescentBiomarkersChildClinical Trials as TopicCognitive DysfunctionDouble-Blind MethodFemaleHumansHydroxymethylglutaryl-CoA Reductase InhibitorsLovastatinMaleOutcome Assessment, Health CareBiomarkersHydroxymethylglutaryl-CoA Reductase InhibitorsLovastatin

Identifiers

PMID31797581
PMCPMC6917317
OpenAlexW2994117439

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.