Evidence map›Paper›PMID 38264537›Full record

ArticleEnvironmental evidence2022

Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews.

Geoff Frampton, Paul Whaley, Micah Bennett, Gary Bilotta, Jean-Lou C M Dorne, Jacqualyn Eales, Katy James, Christian Kohl, Magnus Land, Barbara Livoreil and 5 more

Abstract read
In one paragraph

Article in Environmental evidence, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 33 papers, 4 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
33citing papers in PubMed, 4 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

33 citing papers in PubMed, 4 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Review
  6. Nutrition and Culinary Education in Food Is Medicine Interventions: A Scoping Review.Journal of the Academy of Nutrition and Dietetics · 2026
    Article
  7. Wastewater Treatment Challenges and Circular Reuse for One Health Sustainability: A Review.International journal of environmental research and public health · 2026
    Review
  8. Article
  9. Article
  10. Article
  11. Review
  12. Article
  13. Article
  14. Review
  15. Article
  16. Article
  17. Global landscape of Vibrio parahaemolyticus research: a bibliometric analysis.World journal of microbiology & biotechnology · 2025
    Review
  18. Article
  19. Review
  20. Development of the 10-question household foodwork interactional assessment questionnaire (FIA-Q10).The international journal of behavioral nutrition and physical activity · 2024
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors.

Geoff FramptonSouthampton Health Technology Assessments Centre (SHTAC), Faculty of Medicine, University of Southampton, Southampton, UK.ORCID 0000-0003-2005-0497
Paul WhaleyLancaster Environment Centre, Lancaster University, Lancaster, UK.ORCID 0000-0003-4021-0785
Micah BennettU.S. Environmental Protection Agency, Region 5, Chicago, IL 60604, USA.ORCID 0000-0002-5806-3878
Gary BilottaSchool of Environment and Technology, University of Brighton, Brighton, UK.ORCID 0000-0003-1935-9040
Jean-Lou C M DorneScientific Committee and Emerging Risks Unit, European Food Safety Authority, Via Carlo Magno 1A, 43121 Parma, Italy.ORCID 0000-0003-3305-063X
Jacqualyn EalesEuropean Centre for Environment and Human Health, College of Medicine and Health, University of Exeter, Knowledge Spa, Truro TR1 3HD, UK.ORCID 0000-0002-6693-8697
Katy JamesCentre for Evidence-Based Agriculture, Harper Adams University, Newport, Shropshire TF10 8NB, UK.ORCID 0000-0001-9545-8409
Christian KohlInstitute for Biosafety in Plant Biotechnology (SB), Julius Kühn Institute (JKI) - Federal Research Centre for Cultivated Plants, Berlin, Germany.
Magnus LandFormas, Box 1206, 111 82 Stockholm, Sweden.ORCID 0000-0002-6410-2589
Barbara LivoreilFreelance Consultant, Coopaname, France.ORCID 0000-0003-0230-3737
David MakowskiUMR518, University Paris-Saclay, INRAE, AgroParistech, 16 rue Claude Bernard, 75231 Paris, France.ORCID 0000-0001-6385-3703
Evans MuchiriCentre for Anthropological Research, University of Johannesburg, Johannesburg, South Africa.ORCID 0000-0003-4642-2674
Gillian PetrokofskyOxford Long-Term Ecology Lab, Department of Zoology, University of Oxford, Oxford, UK.ORCID 0000-0001-6748-3869
Nicola RandallCentre for Evidence-Based Agriculture, Harper Adams University, Newport, Shropshire TF10 8NB, UK.ORCID 0000-0002-3023-8488
Kate SchofieldOffice of Research and Development, U.S. Environmental Protection Agency, Washington, DC 20460, USA.

Funding

Intramural EPA EPA999999
6 · The paper itself

Abstract

The internal validity of conclusions about effectiveness or impact in systematic reviews, and of decisions based on them, depends on risk of bias assessments being conducted appropriately. However, a random sample of 50 recently-published articles claiming to be quantitative environmental systematic reviews found 64% did not include any risk of bias assessment, whilst nearly all that did omitted key sources of bias. Other limitations included lack of transparency, conflation of quality constructs, and incomplete application of risk of bias assessments to the data synthesis. This paper addresses deficiencies in risk of bias assessments by highlighting core principles that are required for risk of bias assessments to be fit-for-purpose, and presenting a framework based on these principles to guide review teams on conducting risk of bias assessments appropriately and consistently. The core principles require that risk of bias assessments be Focused, Extensive, Applied and Transparent (FEAT). These principles support risk of bias assessments, appraisal of risk of bias tools, and the development of new tools. The framework follows a Plan-Conduct-Apply-Report approach covering all stages of risk of bias assessment. The scope of this paper is comparative quantitative environmental systematic reviews which address PICO or PECO-type questions including, but not limited to, topic areas such as environmental management, conservation, ecosystem restoration, and analyses of environmental interventions, exposures, impacts and risks.

Indexed as

BiasBlindingCritical appraisalExternal validityInternal validityQuality assessmentRisk of biasSystematic errorValidity

Identifiers

PMID38264537
PMCPMC10805236

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.