ArticleGenome biology2022
Sustained software development, not number of citations or journal choice, is indicative of accurate bioinformatic software.
Article in Genome biology, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Packaging and containerization of computational methods.Nature protocols · 2024Pooled it
- Ten quick tips to SNIFF out sustainable and secure scientific software.PLoS computational biology · 2026Article
- A workflow for statistical analysis and visualization of microbiome omics data using the R microeco package.Nature protocols · 2026Review
- Genetic variants in the CD59 gene: An exploratory study of large genome databases.Transfusion · 2025Article
- Evaluating computational tools for protein-coding sequence detection: Are they up to the task?RNA (New York, N.Y.) · 2025Article
- A bioinformatician, computer scientist, and geneticist lead bioinformatic tool development-which one is better?Bioinformatics advances · 2025Article
- ProEnd: a comprehensive database for identifying HbYX motif-containing proteins across the tree of life.BMC genomics · 2024Article
- ProEnd: A Comprehensive Database for Identifying HbYX Motif-Containing Proteins Across the Tree of Life.bioRxiv : the preprint server for biology · 2024Article
- Article
- A choice, not an obligation : Releasing scientific software as open source should be the responsibility of the authors.EMBO reports · 2024Article
- WebQUAST: online evaluation of genome assemblies.Nucleic acids research · 2023Article
- The Network Zoo: a multilingual package for the inference and analysis of gene regulatory networks.Genome biology · 2023Article
- Cellsnake: a user-friendly tool for single-cell RNA sequencing analysis.GigaScience · 2022Article
- Efficient ancestry and mutation simulation with msprime 1.0.Genetics · 2022Article
- Sustained software development, not number of citations or journal choice, is indicative of accurate bioinformatic software.Genome biology · 2022Article
- On the optimistic performance evaluation of newly introduced bioinformatic methods.Genome biology · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundComputational biology provides software tools for testing and making inferences about biological data. In the face of increasing volumes of data, heuristic methods that trade software speed for accuracy may be employed. We have studied these trade-offs using the results of a large number of independent software benchmarks, and evaluated whether external factors, including speed, author reputation, journal impact, recency and developer efforts, are indicative of accurate software.
resultsWe find that software speed, author reputation, journal impact, number of citations and age are unreliable predictors of software accuracy. This is unfortunate because these are frequently cited reasons for selecting software tools. However, GitHub-derived statistics and high version numbers show that accurate bioinformatic software tools are generally the product of many improvements over time. We also find an excess of slow and inaccurate bioinformatic software tools, and this is consistent across many sub-disciplines. There are few tools that are middle-of-road in terms of accuracy and speed trade-offs.
conclusionsOur findings indicate that accurate bioinformatic software is primarily the product of long-term commitments to software development. In addition, we hypothesise that bioinformatics software suffers from publication bias. Software that is intermediate in terms of both speed and accuracy may be difficult to publish-possibly due to author, editor and reviewer practises. This leaves an unfortunate hole in the literature, as ideal tools may fall into this gap. High accuracy tools are not always useful if they are slow, while high speed is not useful if the results are also inaccurate.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.