Evidence map›Paper›PMID 38555664›Full record

ArticleEnvironment international2024

A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).

Julian P T Higgins, Rebecca L Morgan, Andrew A Rooney, Kyla W Taylor, Kristina A Thayer, Raquel A Silva, Courtney Lemeris, Elie A Akl, Thomas F Bateson, Nancy D Berkman and 16 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Environment international, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 569 papers, 98 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
569citing papers in PubMed, 98 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

569 citing papers in PubMed, 98 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Prolonged and night-time bottle-feeding in relation to early childhood caries: a systematic review.European archives of paediatric dentistry : official journal of the European Academy of Paediatric Dentistry · 2026
    Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Birth outcomes following intrauterine exposure to electronic nicotine delivery systems: a systematic review and meta-analysis.Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco · 2026
    Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

509 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

26 authors.

Julian P T HigginsPopulation Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK; NIHR Bristol Evidence Synthesis Group, University of Bristol, Bristol, UK; NIHR Applied Research Collaboration West (ARC West) at University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, UK. Electronic address: julian.higgins@bristol.ac.uk.
Rebecca L MorganDepartment of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
Andrew A RooneyNational Institute of Environmental Health Sciences, Research Triangle Park, NC, USA.
Kyla W TaylorNational Institute of Environmental Health Sciences, Research Triangle Park, NC, USA.
Kristina A ThayerCenter for Public Health and Environmental Assessment, Chemical & Pollutant Assessment Division, US Environmental Protection Agency, Research Triangle Park, NC, USA.
Raquel A SilvaICF, Reston, VA, USA.
Courtney LemerisICF, Reston, VA, USA.
Elie A AklFaculty of Medicine, American University of Beirut, Riad El-Solh, Lebanon.
Thomas F BatesonCenter for Public Health and Environmental Assessment, Chemical and Pollutant Assessment Division, US Environmental Protection Agency, Washington, DC, USA.
Nancy D BerkmanRTI International, Research Triangle Park, NC, USA.
Barbara S GlennCenter for Public Health and Environmental Assessment, Chemical and Pollutant Assessment Division, US Environmental Protection Agency, Washington, DC, USA.
Asbjørn HróbjartssonCentre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, University of Southern Denmark, Odense, Denmark.
Judy S LaKindLaKind Associates, LLC, Catonsville, MD, USA.
Alexandra McAleenanPopulation Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.
Joerg J MeerpohlInstitute for Evidence in Medicine, Medical Center & Faculty of Medicine, University of Freiburg, Freiburg, Germany; Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany.
Rebecca M NachmanCenter for Public Health and Environmental Assessment, Chemical and Pollutant Assessment Division, US Environmental Protection Agency, Washington, DC, USA.
Julie E ObbagyNutrition Evidence Systematic Review Branch, Center for Nutrition Policy and Promotion, Food and Nutrition Service, US Department of Agriculture, Alexandria, VA, USA.
Annette O'ConnorDepartment of Large Animal Clinical Sciences, College of Veterinary Medicine, Michigan State University, East Lansing, MI, USA.
Elizabeth G RadkeCenter for Public Health and Environmental Assessment, Chemical and Pollutant Assessment Division, US Environmental Protection Agency, Washington, DC, USA.
Jelena SavovićPopulation Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK; NIHR Bristol Evidence Synthesis Group, University of Bristol, Bristol, UK; NIHR Applied Research Collaboration West (ARC West) at University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, UK.
Holger J SchünemannCochrane Canada and McMaster GRADE Centres, McMaster University, Hamilton, ON, Canada.
Beverley SheaOttawa Hospital Research Institute, Ottawa, ON, Canada.
Kate TillingPopulation Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK; NIHR Applied Research Collaboration West (ARC West) at University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, UK; MRC Integrative Epidemiology Unit, University of Bristol, Bristol, UK; NIHR Bristol Biomedical Research Centre, Bristol, UK.
Jos VerbeekCochrane Work, Department of Public and Occupational Health, Academic Medical Centers Amsterdam, University of Amsterdam, Amsterdam, the Netherlands.
Meera ViswanathanRTI International, Research Triangle Park, NC, USA.
Jonathan A C SternePopulation Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK; Health Data Research UK South-West, Bristol, UK.

Funding

Scientific Cyberinfrastructure Research ProgramZIAES103385 · NIEHS · NATIONAL INSTITUTE OF ENVIRONMENTAL HEALTH SCIENCES · PI WALKER, VICKIE · 2022 to 2025
$17.4M
Consumer Products and Therapeutics Research ProgramZIAES103379 · NIEHS · NATIONAL INSTITUTE OF ENVIRONMENTAL HEALTH SCIENCES · PI TAYLOR, KYLA · 2022 to 2025
$9.9M
Cancer Research UK 29019Intramural EPA EPA999999NIEHS NIH HHS HHSN273201600015C
6 · The paper itself

Abstract

backgroundObservational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies.

objectiveTo develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND

resultsROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias.

conclusionROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.

Indexed as

BiasEnvironmental ExposureCohort StudiesEpidemiologic StudiesFollow-Up StudiesHumansObservational Studies as TopicRisk AssessmentConfoundingEnvironmentalEpidemiologyExposureMisclassification/measurement biasRisk of biasSelection bias

Identifiers

PMID38555664
PMCPMC11098530

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.