Evidence map›Paper›PMID 33121530›Full record

SynthesisSystematic reviews2020

Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools.

Stephanie M Eick, Dana E Goin, Nicholas Chartres, Juleen Lam, Tracey J Woodruff

Abstract readSystematic Review
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in Systematic reviews, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 52 papers, 16 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
52citing papers in PubMed, 16 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

52 citing papers in PubMed, 16 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Psychotropic drug repurposing for COVID-19: A Systematic Review and Meta-Analysis.European neuropsychopharmacology : the journal of the European College of Neuropsychopharmacology · 2023
    Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Review
  18. Review
  19. Review
  20. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

5 authors.

Stephanie M EickProgram on Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, USA.
Dana E GoinProgram on Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, USA.
Nicholas ChartresProgram on Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, USA.
Juleen LamProgram on Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, USA.
Tracey J WoodruffProgram on Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, USA. tracey.woodruff@ucsf.edu.ORCID 0000-0003-3622-1297

Funding

The Illinois Kids Development Study ECHO Pregnancy and Pediatric CohortUH3OD023272 · OD · UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN · PI Andrea Aguiar, Susan L Schantz · 2018 to 2026
$18.6M
The Illinois Kids Development Study ECHO Pregnancy and Pediatric CohortUG3OD023272 · OD · UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN · PI SCHANTZ, SUSAN L · 2016 to 2024
$7.2M
Transdisciplinary Approach to Rapidly Identify Reproductive Toxics in Pregnant Women and NewbornsR01ES027051 · NIEHS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI Dimitri Abrahamsson, Tracey J. Woodruff · 2016 to 2026
$6.6M
The UCSF Pregnancy Exposures to Environmental Chemicals (PEEC) Children's CenterP01ES022841 · NIEHS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI WOODRUFF, TRACEY J. · 2013 to 2018
$3.9M
NIEHS NIH HHS P01 ES022841NIEHS NIH HHS R01 ES027051NIH HHS UG3 OD023272NIH HHS UH3 OD023272
6 · The paper itself

Abstract

backgroundSystematic reviews are increasingly prevalent in environmental health due to their ability to synthesize evidence while reducing bias. Different systematic review methods have been developed by the US National Toxicology Program's Office of Health Assessment and Translation (OHAT), the US Environmental Protection Agency's (EPA) Integrated Risk Information System (IRIS), and by the US EPA under the Toxic Substances Control Act (TSCA), including the approach to assess risk of bias (ROB), one of the most vital steps which is used to evaluate internal validity of the studies. Our objective was to compare the performance of three tools (OHAT, IRIS, TSCA) in assessing ROB.

methodsWe selected a systematic review on polybrominated diphenyl ethers and intelligence quotient and/or attention deficit hyperactivity disorder because it had been endorsed by the National Academy of Sciences. Two reviewers followed verbatim instructions from the tools and independently applied each tool to assess ROB in 15 studies previously identified. We documented the time to apply each tool and the impact the ROB ratings for each tool had on the final rating of the quality of the overall body of evidence.

resultsThe time to complete the ROB assessments varied widely (mean = 20, 32, and 40 min per study for the OHAT, IRIS, and TSCA tools, respectively). All studies were rated overall "low" or "uninformative" using IRIS, due to "deficient" or "critically deficient" ratings in one or two domains. Similarly, all studies were rated "unacceptable" using the TSCA tool because of one "unacceptable" rating in a metric related to statistical power. Approximately half of the studies had "low" or "probably low ROB" ratings across all domains with the OHAT and Navigation Guide tools.

conclusionsTools that use overall ROB or study quality ratings, such as IRIS and TSCA, may reduce the available evidence to assess the harms of environmental exposures by erroneously excluding studies, which leads to inaccurate conclusions about the quality of the body of evidence. We recommend using ROB tools that circumvents these issues, such as OHAT and Navigation Guide. SYSTEMATIC REVIEW REGISTRATION: This review has not been registered as it is not a systematic review.

Indexed as

Environmental HealthResearch DesignBiasHumansResearch ReportRisk AssessmentCritical appraisalEvidence evaluationQuality assessmentRisk assessmentRisk of biasSystematic review

Identifiers

PMID33121530
PMCPMC7596989

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.