Evidence map›Paper›PMID 34273858›Full record

ArticleJournal of clinical virology : the official publication of the Pan American Society for Clinical Virology2021

Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples.

Jutte J C de Vries, Julianne R Brown, Nicole Fischer, Igor A Sidorov, Sofia Morfopoulou, Jiabin Huang, Bas B Oude Munnink, Arzu Sayiner, Alihan Bulgurcu, Christophe Rodriguez and 20 more

Open access · hybridAbstract read
In one paragraph

Article in Journal of clinical virology : the official publication of the Pan American Society for Clinical Virology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 44 papers.

0numbers the graph read from it
0cells of the map it votes in
44citing papers in PubMed
8.9field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

44 citing papers in PubMed, 73 citations in OpenAlex.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Review
  6. Article
  7. Article
  8. Review
  9. Review
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Review
  18. Review
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

30 authors at 16 institutions in 11 countries.

Jutte J C de VriesClinical Microbiological Laboratory, Department of Medical Microbiology, Leiden University Medical Center, Leiden, the Netherlands. Electronic address: jjcdevries@lumc.nl.
Julianne R BrownMicrobiology, Virology and Infection Prevention & Control, Great Ormond Street Hospital for Children NHS Foundation Trust, London, United Kingdom. Electronic address: julianne.brown@gosh.nhs.uk.
Nicole FischerUniversity Medical Center Hamburg-Eppendorf, UKE Institute for Medical Microbiology, Virology and Hygiene, Germany. Electronic address: nfischer@uke.de.
Igor A SidorovClinical Microbiological Laboratory, Department of Medical Microbiology, Leiden University Medical Center, Leiden, the Netherlands. Electronic address: I.A.Sidorov@lumc.nl.
Sofia MorfopoulouDivision of Infection and Immunity, University College London, London, United Kingdom. Electronic address: sofia.morfopoulou.10@ucl.ac.uk.
Jiabin HuangUniversity Medical Center Hamburg-Eppendorf, UKE Institute for Medical Microbiology, Virology and Hygiene, Germany. Electronic address: j.huang@uke.de.
Bas B Oude MunninkViroscience, Erasmus Medical Center, Rotterdam, the Netherlands. Electronic address: b.oudemunnink@erasmusmc.nl.
Arzu SayinerDokuz Eylul University, Medical Faculty, Izmir, Turkey. Electronic address: arzu.sayiner@deu.edu.tr.
Alihan BulgurcuHospital Henri Mondor, Paris, France.
Christophe RodriguezHospital Henri Mondor, Paris, France. Electronic address: christophe.rodriguez@aphp.fr.
Guillaume GricourtHospital Henri Mondor, Paris, France. Electronic address: guillaume.gricourt@aphp.fr.
Els KeyaertsLaboratory of Clinical and Epidemiological Virology (Rega Institute), KU Leuven, Belgium. Electronic address: els.keyaerts@kuleuven.be.
Leen BellerLaboratory of Clinical and Epidemiological Virology (Rega Institute), KU Leuven, Belgium. Electronic address: leen.beller@kuleuven.be.
Claudia BachofenInstitute of Virology, University of Zurich, Switzerland. Electronic address: claudia.bachofen@uzh.ch.
Jakub KubackiInstitute of Virology, University of Zurich, Switzerland. Electronic address: jakub.kubacki@uzh.ch.
Cordey SamuelLaboratory of Virology, University Hospitals of Geneva, Geneva, Switzerland. Electronic address: Samuel.Cordey@hcuge.ch.
Laubscher FlorianLaboratory of Virology, University Hospitals of Geneva, Geneva, Switzerland. Electronic address: Florian.Laubscher@hcuge.ch.
Schmitz DennisRIVM National Institute for Public Health and Environment, Bilthoven, the Netherlands. Electronic address: Dennis.Schmitz@RIVM.nl.
Martin BeerFriedrich-Loeffler-Institute, Institute of Diagnostic Virology, Greifswald, Germany. Electronic address: martin.beer@fli.de.
Dirk HoeperFriedrich-Loeffler-Institute, Institute of Diagnostic Virology, Greifswald, Germany. Electronic address: dirk.hoeper@fli.de.
Michael HuberInstitute of Medical Virology, University of Zurich, Switzerland. Electronic address: huber.michael@virology.uzh.ch.
Verena KufnerInstitute of Medical Virology, University of Zurich, Switzerland. Electronic address: kufner.verena@virology.uzh.ch.
Maryam ZaheriInstitute of Medical Virology, University of Zurich, Switzerland. Electronic address: zaheri.maryam@virology.uzh.ch.
Aitana LebrandSwiss Institute of Bioinformatics, Geneva, Switzerland. Electronic address: aitana.lebrand@sib.swiss.
Anna PapaDepartment of Microbiology, Medical School, Aristotle University of Thessaloniki, Greece. Electronic address: annap@auth.gr.
Sander van BoheemenViroscience, Erasmus Medical Center, Rotterdam, the Netherlands. Electronic address: s.vanboheemen@erasmusmc.nl.
Aloys C M KroesClinical Microbiological Laboratory, Department of Medical Microbiology, Leiden University Medical Center, Leiden, the Netherlands. Electronic address: A.C.M.Kroes@lumc.nl.
Judith BreuerMicrobiology, Virology and Infection Prevention & Control, Great Ormond Street Hospital for Children NHS Foundation Trust, London, United Kingdom; Division of Infection and Immunity, University College London, London, United Kingdom. Electronic address: breuej@gosh.nhs.uk.
F Xavier Lopez-LabradorVirology Laboratory, Genomics and Health Area, Center for Public Health Research (FISABIO-Public Health), Generalitat Valenciana and Microbiology & Ecology Department, University of Valencia, Spain; CIBERESP, Instituto de Salud Carlos III, Spain. Electronic address: F.Xavier.Lopez@uv.es.
Eric C J ClaasClinical Microbiological Laboratory, Department of Medical Microbiology, Leiden University Medical Center, Leiden, the Netherlands. Electronic address: E.C.J.Claas@lumc.nl.
Leiden University Medical Center · NLInstitute of Virology of the Slovak Academy of Sciences · SKDokuz Eylül University · TRErasmus MC · NLFriedrich-Loeffler-Institut · DEGreat Ormond Street Hospital for Children NHS Foundation Trust · GBHôpitaux Universitaires Henri-Mondor · FRRega Institute for Medical Research · BEUniversität Hamburg · DEUniversity Hospital of Geneva · CHUniversity of Zurich · CHAristotle University of Thessaloniki · GRInstitute of Infection and Immunity · CANational Institute for Public Health and the Environment · NLSIB Swiss Institute of Bioinformatics · CHUniversitat de València · ES

Funding

Wellcome Trust 206478Wellcome Trust 206478/Z/17/Z
6 · The paper itself

Abstract

introductionMetagenomic sequencing is increasingly being used in clinical settings for difficult to diagnose cases. The performance of viral metagenomic protocols relies to a large extent on the bioinformatic analysis. In this study, the European Society for Clinical Virology (ESCV) Network on NGS (ENNGS) initiated a benchmark of metagenomic pipelines currently used in clinical virological laboratories.

methodsMetagenomic datasets from 13 clinical samples from patients with encephalitis or viral respiratory infections characterized by PCR were selected. The datasets were analyzed with 13 different pipelines currently used in virological diagnostic laboratories of participating ENNGS members. The pipelines and classification tools were: Centrifuge, DAMIAN, DIAMOND, DNASTAR, FEVIR, Genome Detective, Jovian, MetaMIC, MetaMix, One Codex, RIEMS, VirMet, and Taxonomer. Performance, characteristics, clinical use, and user-friendliness of these pipelines were analyzed.

resultsOverall, viral pathogens with high loads were detected by all the evaluated metagenomic pipelines. In contrast, lower abundance pathogens and mixed infections were only detected by 3/13 pipelines, namely DNASTAR, FEVIR, and MetaMix. Overall sensitivity ranged from 80% (10/13) to 100% (13/13 datasets). Overall positive predictive value ranged from 71-100%. The majority of the pipelines classified sequences based on nucleotide similarity (8/13), only a minority used amino acid similarity, and 6 of the 13 pipelines assembled sequences de novo. No clear differences in performance were detected that correlated with these classification approaches. Read counts of target viruses varied between the pipelines over a range of 2-3 log, indicating differences in limit of detection.

conclusionA wide variety of viral metagenomic pipelines is currently used in the participating clinical diagnostic laboratories. Detection of low abundant viral pathogens and mixed infections remains a challenge, implicating the need for standardization and validation of metagenomic analysis for clinical diagnostic use. Future studies should address the selective effects due to the choice of different reference viral databases.

Indexed as

Computational BiologyVirusesBenchmarkingHigh-Throughput Nucleotide SequencingHumansMetagenomicsBenchmarkBioinformatic pipelinesViral metagenomics

Identifiers

PMID34273858
PMCPMC7615111
OpenAlexW3162965417

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.