Evidence map›Paper›PMID 38684791›Full record

ArticleScientific reports2024

Variability and bias in microbiome metagenomic sequencing: an interlaboratory study comparing experimental protocols.

Samuel P Forry, Stephanie L Servetas, Jason G Kralj, Keng Soh, Michalis Hadjithomas, Raul Cano, Martha Carlin, Maria G de Amorim, Benjamin Auch, Matthew G Bakker and 23 more

Open access · goldAbstract readComparative Study
In one paragraph

Article in Scientific reports, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 46 papers.

0numbers the graph read from it
0cells of the map it votes in
46citing papers in PubMed
9.1field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

46 citing papers in PubMed, 40 citations in OpenAlex.

  1. Review
  2. Review
  3. Review
  4. Article
  5. Review
  6. Article
  7. Review
  8. Review
  9. Article
  10. Review
  11. Article
  12. Article
  13. The gut-joint axis in osteoarthritis.Nature reviews. Rheumatology · 2026
    Review
  14. Article
  15. Article
  16. Article
  17. Review
  18. Article
  19. Review
  20. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

33 authors at 20 institutions in 10 countries.

Samuel P ForryComplex Microbial Systems Group, National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA. sam.forry@nist.gov.
Stephanie L ServetasComplex Microbial Systems Group, National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA.
Jason G KraljComplex Microbial Systems Group, National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA.
Keng SohNovo Nordisk, Copenhagen, Denmark.
Michalis HadjithomasLifeMine Therapeutics, Cambridge Discovery Park, 30 Acorn Park Drive, Cambridge, MA, 02140, USA.
Raul CanoThe BioCollective, LLC, 5650 Washington Street, Suite C9, Denver, CO, 80216, USA.
Martha CarlinThe BioCollective, LLC, 5650 Washington Street, Suite C9, Denver, CO, 80216, USA.
Maria G de AmorimLaboratory of Medical Genomics, A. C. Camargo Cancer Center, Sao Paulo, SP, 01508-010, Brazil.
Benjamin AuchUniversity of Minnesota Genomics Center, Minneapolis, MN, 55455, USA.
Matthew G BakkerDepartment of Microbiology, University of Manitoba, Winnipeg, MB, R3T 2N2, Canada.
Thais F BartelliLaboratory of Medical Genomics, A. C. Camargo Cancer Center, Sao Paulo, SP, 01508-010, Brazil.
Juan P BustamanteLaboratorio de Investigación, Desarrollo y Transferencia de la Facultad de Ingeniería de la Universidad Austral (LIDTUA), CIC-Austral, Pilar, Argentina.
Ignacio CassolLaboratorio de Investigación, Desarrollo y Transferencia de la Facultad de Ingeniería de la Universidad Austral (LIDTUA), CIC-Austral, Pilar, Argentina.
Mauricio ChalitaCJ Bioscience, Seoul, South Korea.
Emmanuel Dias-NetoLaboratory of Medical Genomics, A. C. Camargo Cancer Center, Sao Paulo, SP, 01508-010, Brazil.
Aaron Del DucaOMX Advisors, Inc., Ottawa, Canada.
Daryl M GohlUniversity of Minnesota Genomics Center, Minneapolis, MN, 55455, USA.
Jekaterina KazantsevaCenter of Food and Fermentation Technologies (TFTAK), Mäealuse 2/4, 12618, Tallinn, Estonia.
Muyideen T HarunaBioenvironmental Program, Morgan State University, Baltimore, MD, USA.
Peter MenzelLabor Berlin Charité Vivantes GmbH, Sylter Str. 2, 13353, Berlin, Germany.
Bruno S ModaLaboratory of Medical Genomics, A. C. Camargo Cancer Center, Sao Paulo, SP, 01508-010, Brazil.
Lorieza Neuberger-CastilloIntegrated Biobank of Luxembourg (IBBL), Luxembourg Institute of Health (LIH), Dudelange, Luxembourg.
Diana N NunesLaboratory of Medical Genomics, A. C. Camargo Cancer Center, Sao Paulo, SP, 01508-010, Brazil.
Isha R PatelCenter for Food Safety and Applied Nutrition, Office of Applied Research and Safety Assessment, U. S. Food and Drug Administration, Laurel, MD, 20708, USA.
Rodrigo D PeraltaLaboratorio de Investigación, Desarrollo y Transferencia de la Facultad de Ingeniería de la Universidad Austral (LIDTUA), CIC-Austral, Pilar, Argentina.
Adrien SaliouOMICS Hub, BIOASTER, Microbiology Research Institute, Lyon, France.
Rolf SchwarzerLabor Berlin Charité Vivantes GmbH, Sylter Str. 2, 13353, Berlin, Germany.
Samantha SevillaCenter for Cancer Research, CCR Collaborative Bioinformatics Resource, National Cancer Institute, National Institutes of Health, Bethesda, MD, 20892, USA.
Isabella K T M TakenakaLaboratory of Medical Genomics, A. C. Camargo Cancer Center, Sao Paulo, SP, 01508-010, Brazil.
Jeremy R WangDepartment of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA.
Rob KnightDepartments of Pediatrics, Bioengineering and Computer Science & Engineering, and Center for Microbiome Innovation, University of California at San Diego, 9500 Gilman Drive, MC 0763, La Jolla, CA, 92093-0763, USA.
Dirk GeversSeed Health, 2100 Abbot Kinney Blvd, Venice, CA, 90291-7003, USA.
Scott A JacksonComplex Microbial Systems Group, National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA.
AC Camargo Hospital · BRNational Institute of Standards and Technology · USBiocon (Switzerland) · CHUniversity of Minnesota · USVivantes Klinikum · DEArk Medical Center · USAustral University · ARBIOASTER · FRCenter for Food Safety and Applied Nutrition · USConsejo Nacional de Investigaciones Científicas y Técnicas · ARHoneywell (United States) · USLeidos (United States) · USLuxembourg Institute of Health · LUMorgan State University · USNovo Nordisk (Denmark) · DKTallinn University of Technology · EEUniversidad Nacional de Entre Ríos · ARUniversity of California San Diego · USUniversity of Manitoba · CAUniversity of North Carolina at Chapel Hill · US

Funding

Advanced sequencing as a novel diagnostic tool to discover strain-level variation and function of mucosal-adherent bacteria contributing to IBDK01DK119582 · NIDDK · UNIV OF NORTH CAROLINA CHAPEL HILL · PI WANG, JEREMY R · 2019 to 2023
$754k
NIDDK NIH HHS K01 DK119582
6 · The paper itself

Abstract

Several studies have documented the significant impact of methodological choices in microbiome analyses. The myriad of methodological options available complicate the replication of results and generally limit the comparability of findings between independent studies that use differing techniques and measurement pipelines. Here we describe the Mosaic Standards Challenge (MSC), an international interlaboratory study designed to assess the impact of methodological variables on the results. The MSC did not prescribe methods but rather asked participating labs to analyze 7 shared reference samples (5 × human stool samples and 2 × mock communities) using their standard laboratory methods. To capture the array of methodological variables, each participating lab completed a metadata reporting sheet that included 100 different questions regarding the details of their protocol. The goal of this study was to survey the methodological landscape for microbiome metagenomic sequencing (MGS) analyses and the impact of methodological decisions on metagenomic sequencing results. A total of 44 labs participated in the MSC by submitting results (16S or WGS) along with accompanying metadata; thirty 16S rRNA gene amplicon datasets and 14 WGS datasets were collected. The inclusion of two types of reference materials (human stool and mock communities) enabled analysis of both MGS measurement variability between different protocols using the biologically-relevant stool samples, and MGS bias with respect to ground truth values using the DNA mixtures. Owing to the compositional nature of MGS measurements, analyses were conducted on the ratio of Firmicutes: Bacteroidetes allowing us to directly apply common statistical methods. The resulting analysis demonstrated that protocol choices have significant effects, including both bias of the MGS measurement associated with a particular methodological choices, as well as effects on measurement robustness as observed through the spread of results between labs making similar methodological choices. In the analysis of the DNA mock communities, MGS measurement bias was observed even when there was general consensus among the participating laboratories. This study was the result of a collaborative effort that included academic, commercial, and government labs. In addition to highlighting the impact of different methodological decisions on MGS result comparability, this work also provides insights for consideration in future microbiome measurement study design.

Indexed as

FecesMetagenomicsMicrobiotaRNA, Ribosomal, 16SBacteriaBiasGastrointestinal MicrobiomeHigh-Throughput Nucleotide SequencingHumansMetagenomeSequence Analysis, DNARNA, Ribosomal, 16S

Identifiers

PMID38684791
PMCPMC11059151
OpenAlexW4396242074

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.