Evidence map›Paper›PMID 40451243›Full record

ArticleGigaScience2025

Analysis-ready VCF at Biobank scale using Zarr.

Eric Czech, Will Tyler, Tom White, Ben Jeffery, Timothy R Millar, Benjamin Elsworth, Jérémy Guez, Jonny Hancox, Konrad J Karczewski, Alistair Miles and 6 more

Abstract read
In one paragraph

Article in GigaScience, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 6 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
6citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

6 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Review
  4. Article
  5. A Pandemic-Scale Ancestral Recombination Graph for SARS-CoV-2.bioRxiv : the preprint server for biology · 2025
    Article
  6. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

16 authors.

Eric CzechOpen Athena AI Foundation, 1245 Broadway, 16th Floor, New York, NY 10001, USA.ORCID 0000-0002-4254-4255
Will TylerIndependent researcher.
Tom WhiteTom White Consulting Ltd.
Ben JefferyBig Data Institute, Li Ka Shing Centre for Health Information and Discovery, University of Oxford, Oxford, OX3 7LF, UK.ORCID 0000-0002-1982-6801
Timothy R MillarThe New Zealand Institute for Plant & Food Research Ltd, 74 Gerald Street,  Lincoln 7608, New Zealand.ORCID 0000-0002-5142-8811
Benjamin ElsworthOur Future Health, 2 New Bailey, 6 Stanley Street, Manchester M3 5GS, UK.ORCID 0000-0001-7328-4233
Jérémy GuezProgram in Medical and Population Genetics, Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA.ORCID 0009-0007-6406-5187
Jonny HancoxNVIDIA Ltd, 100 Brook Drive, Green Park, Reading RG2 6UJ, UK.ORCID 0009-0003-5799-5299
Konrad J KarczewskiProgram in Medical and Population Genetics, Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA.ORCID 0000-0003-2878-4671
Alistair MilesGenomic Surveillance Unit, Wellcome Sanger Institute, Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SA, UK.ORCID 0000-0001-9018-4680
Sam TallmanGenomics England, One Canada Square, London, E14 5AB, UK.ORCID 0000-0001-7183-6276
Per UnnebergDepartment of Cell and Molecular Biology, National Bioinformatics Infrastructure Sweden, Science for Life Laboratory, Uppsala University, Husargatan 3, SE-752 37 Uppsala, Sweden.ORCID 0000-0001-5735-3315
Rafal WojdylaOpen Athena AI Foundation, 1245 Broadway, 16th Floor, New York, NY 10001, USA.ORCID 0009-0005-0735-7090
Shadi ZabadSchool of Computer Science, McGill University, Montreal, QC, H3A 2A7, Canada.ORCID 0000-0002-8003-9284
Jeff HammerbacherOpen Athena AI Foundation, 1245 Broadway, 16th Floor, New York, NY 10001, USA.ORCID 0000-0001-6596-8563
Jerome KelleherBig Data Institute, Li Ka Shing Centre for Health Information and Discovery, University of Oxford, Oxford, OX3 7LF, UK.ORCID 0000-0002-7894-5253

Funding

Scaling up computational genomics with tree sequencesR01HG012473 · NHGRI · UNIVERSITY OF OREGON · PI PETER Lochhead RALPH · 2023 to 2026
$2.3M
Scaling up computational genomics with tree sequencesR56HG011395 · NHGRI · UNIVERSITY OF OREGON · PI RALPH, PETER LOCHHEAD · 2021 to 2021
$557k
Gates Foundation INV-001927Knut and Alice Wallenberg Foundation KAW 2017.0003Knut and Alice Wallenberg Foundation KAW 2020.0239National Bioinformatics Infrastructure Sweden (NBIS)NHGRI NIH HHS R01 HG012473NHGRI NIH HHS R56 HG011395Robertson Foundation and NIH HG011395Robertson Foundation and NIH HG012473SciLifeLab & Wallenberg Data Driven Life Science ProgramThe New Zealand Institute for Plant & Food Research Ltd Kiwifruit Royalty Investment ProgrammeWellcome Trust
6 · The paper itself

Abstract

backgroundVariant Call Format (VCF) is the standard file format for interchanging genetic variation data and associated quality control metrics. The usual row-wise encoding of the VCF data model (either as text or packed binary) emphasizes efficient retrieval of all data for a given variant, but accessing data on a field or sample basis is inefficient. The Biobank-scale datasets currently available consist of hundreds of thousands of whole genomes and hundreds of terabytes of compressed VCF. Row-wise data storage is fundamentally unsuitable and a more scalable approach is needed.

resultsZarr is a format for storing multidimensional data that is widely used across the sciences, and is ideally suited to massively parallel processing. We present the VCF Zarr specification, an encoding of the VCF data model using Zarr, along with fundamental software infrastructure for efficient and reliable conversion at scale. We show how this format is far more efficient than standard VCF-based approaches, and competitive with specialized methods for storing genotype data in terms of compression ratios and single-threaded calculation performance. We present case studies on subsets of 3 large human datasets (Genomics England: $n$=78,195; Our Future Health: $n$=651,050; All of Us: $n$=245,394) along with whole genome datasets for Norway Spruce ($n$=1,063) and SARS-CoV-2 ($n$=4,484,157). We demonstrate the potential for VCF Zarr to enable a new generation of high-performance and cost-effective applications via illustrative examples using cloud computing and GPUs.

conclusionsLarge row-encoded VCF files are a major bottleneck for current research, and storing and processing these files incurs a substantial cost. The VCF Zarr specification, building on widely used, open-source technologies, has the potential to greatly reduce these costs, and may enable a diverse ecosystem of next-generation tools for analysing genetic variation data directly from cloud-based object stores, while maintaining compatibility with existing file-oriented workflows.

Indexed as

Biological Specimen BanksGenetic VariationSoftwareCOVID-19Databases, GeneticGenome, HumanHumansSARS-CoV-2analysis-ready dataVariant Call FormatZarr

Identifiers

PMID40451243
PMCPMC12127038

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.