Evidence map›Paper›PMID 37731657›Full record

ArticleArXiv2024

Evaluation of large language models for discovery of gene set function.

Mengzhou Hu, Sahar Alkhairy, Ingoo Lee, Rudolf T Pillich, Dylan Fong, Kevin Smith, Robin Bachelder, Trey Ideker, Dexter Pratt

Abstract readPreprint
PubMed Full text at PMC
In one paragraph

Article in ArXiv, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

9 authors.

Mengzhou HuDepartment of Medicine, University of California San Diego, La Jolla, California, USA.
Sahar AlkhairyDepartment of Computer Science and Engineering, University of California San Diego, La Jolla, California, USA.
Ingoo LeeDepartment of Medicine, University of California San Diego, La Jolla, California, USA.
Rudolf T PillichDepartment of Medicine, University of California San Diego, La Jolla, California, USA.
Dylan FongDepartment of Medicine, University of California San Diego, La Jolla, California, USA.
Kevin SmithDepartment of Physics, University of California San Diego, La Jolla, California, USA.
Robin BachelderDepartment of Medicine, University of California San Diego, La Jolla, California, USA.
Trey IdekerDepartment of Medicine, University of California San Diego, La Jolla, California, USA.
Dexter PrattDepartment of Medicine, University of California San Diego, La Jolla, California, USA.

Funding

Bridge2AI: Cell Maps for AI (CM4AI) Data Generation ProjectOT2OD032742 · OD · UNIVERSITY OF CALIFORNIA, SAN DIEGO · PI Jean-Christophe Bélisle-Pipon, TIMOTHY W CLARK · 2022 to 2026
$21.5M
The Psychiatric Cell Map Initiative: Connecting Genomics, Subcellular Networks, and Higher Order PhenotypesU01MH115747 · NIMH · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI KAMPMANN, MARTIN, KROGAN, NEVAN J · 2018 to 2022
$19.6M
TR&D 3 - Network Guided Machine LearningP41GM103504 · NIGMS · UNIVERSITY OF CALIFORNIA, SAN DIEGO · PI BADER, GARY D. · 2012 to 2024
$17.3M
The Cancer Cell Map Initiative v2.0U54CA274502 · NCI · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI Nevan J Krogan · 2022 to 2026
$14.2M
CYTOSCAPE: AN ECOSYSTEM FOR NETWORK GENOMICSU24HG012107 · NHGRI · UNIVERSITY OF CALIFORNIA, SAN DIEGO · PI Jing Chen, PIETER C DORRESTEIN · 2022 to 2026
$6.5M
The Network Data Exchange - a Network Commons for BiologistsU24CA269436 · NCI · UNIVERSITY OF CALIFORNIA, SAN DIEGO · PI Dexter Pratt · 2023 to 2026
$3.5M
NCI NIH HHS U24 CA269436NCI NIH HHS U54 CA274502NHGRI NIH HHS U24 HG012107NIGMS NIH HHS P41 GM103504NIH HHS OT2 OD032742NIMH NIH HHS U01 MH115747
6 · The paper itself

Abstract

Gene set analysis is a mainstay of functional genomics, but it relies on curated databases of gene functions that are incomplete. Here we evaluate five Large Language Models (LLMs) for their ability to discover the common biological functions represented by a gene set, substantiated by supporting rationale, citations and a confidence assessment. Benchmarking against canonical gene sets from the Gene Ontology, GPT-4 confidently recovered the curated name or a more general concept (73% of cases), while benchmarking against random gene sets correctly yielded zero confidence. Gemini-Pro and Mixtral-Instruct showed ability in naming but were falsely confident for random sets, whereas Llama2-70b had poor performance overall. In gene sets derived from 'omics data, GPT-4 identified novel functions not reported by classical functional enrichment (32% of cases), which independent review indicated were largely verifiable and not hallucinations. The ability to rapidly synthesize common gene functions positions LLMs as valuable 'omics assistants.

Identifiers

PMID37731657
PMCPMC10508824

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.