ArticleBriefings in bioinformatics2025
Benchmarking large language models for genomic knowledge with GeneTuring.
Article in Briefings in bioinformatics, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 15 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
15 citing papers in PubMed.
- Governing AI for Pharmacovigilance in Low-Income Countries: Systems Perspective.Journal of medical Internet research · 2026Article
- Eras of bioinformatics technologies from command-line interfaces to artificial intelligence (AI) chatbots.Briefings in bioinformatics · 2026Review
- GeneGenie: enhancing biomedical question-answering with agentic graphs.Briefings in bioinformatics · 2026Article
- Med.ai ASK: an agentic system for biomedical question answering.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Evaluating open LLMs for agentic analysis orchestration in a typical biomedical lab.bioRxiv : the preprint server for biology · 2026Article
- Lense: Optimizing data preprocessing in single-cell omics using LLMs.bioRxiv : the preprint server for biology · 2026Article
- Lense: optimizing data preprocessing in single-cell omics using large language models.Briefings in bioinformatics · 2026Article
- From Prompt to Drug: Toward Pharmaceutical Superintelligence.ACS central science · 2026Review
- Federated knowledge retrieval elevates large language model performance on biomedical benchmarks.GigaScience · 2026Article
- AI-driven discovery in protein science for immunology and infectious disease research.Frontiers in bioinformatics · 2026Review
- Prompt injection of OpenAI custom GPTs leaks informatics secrets.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2026Article
- GPTAnno: Ontology-tree-guided hierarchical cell type annotation based on GPT models for single-cell data.bioRxiv : the preprint server for biology · 2025Article
- Recent advances in antibody optimization based on deep learning methods.Journal of Zhejiang University. Science. B · 2025Review
- Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Models.ArXiv · 2025Article
- Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Models.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
Corrections and comments
- Update of
Authors and funding
4 authors.
Funding
Abstract
Large language models (LLMs) show promise in biomedical research, but their effectiveness for genomic inquiry remains unclear. We developed GeneTuring, a benchmark consisting of 16 genomics tasks with 1600 curated questions, and manually evaluated 48 000 answers from 10 LLM configurations, including GPT-4o (via API, ChatGPT with web access, and a custom Generative Pretrained Transformer (GPT) setup), GPT-3.5, Claude 3.5, Gemini Advanced, GeneGPT (both slim and full), BioGPT, and BioMedLM. A custom GPT-4o configuration integrated with National Center for Biotechnology Information (NCBI) Application Programming Interfaces (APIs), developed in this study as SeqSnap, achieved the best overall performance. GPT-4o with web access and GeneGPT demonstrated complementary strengths. Our findings highlight both the promise and current limitations of LLMs in genomics, and emphasize the value of combining LLMs with domain-specific tools for robust genomic intelligence. GeneTuring offers a key resource for benchmarking and improving LLMs in biomedical research.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.