Evidence map›Paper›PMID 37957207›Full record

ArticleNature communications2023

CLOOME: contrastive learning unlocks bioimaging databases for queries with chemical structures.

Ana Sanchez-Fernandez, Elisabeth Rumetshofer, Sepp Hochreiter, Günter Klambauer

Abstract read
In one paragraph

Article in Nature communications, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 29 papers.

0numbers the graph read from it
0cells of the map it votes in
29citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

29 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Review
  6. Article
  7. Review
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Review
  14. Article
  15. Review
  16. Article
  17. Integrating Biological Knowledge for Robust Microscopy Image Profiling onProceedings. IEEE International Conference on Computer Vision · 2025
    Article
  18. MorphoDiff: Cellular Morphology Painting with Diffusion Models.bioRxiv : the preprint server for biology · 2024
    Article
  19. Decoding phenotypic screening: A comparative analysis of image representations.Computational and structural biotechnology journal · 2024
    Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors.

Ana Sanchez-FernandezELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Linz, Austria.
Elisabeth RumetshoferELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Linz, Austria.
Sepp HochreiterELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Linz, Austria.ORCID 0000-0001-7449-2528
Günter KlambauerELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Linz, Austria. klambauer@ml.jku.at.ORCID 0000-0003-2861-5552

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

The field of bioimage analysis is currently impacted by a profound transformation, driven by the advancements in imaging technologies and artificial intelligence. The emergence of multi-modal AI systems could allow extracting and utilizing knowledge from bioimaging databases based on information from other data modalities. We leverage the multi-modal contrastive learning paradigm, which enables the embedding of both bioimages and chemical structures into a unified space by means of bioimage and molecular structure encoders. This common embedding space unlocks the possibility of querying bioimaging databases with chemical structures that induce different phenotypic effects. Concretely, in this work we show that a retrieval system based on multi-modal contrastive learning is capable of identifying the correct bioimage corresponding to a given chemical structure from a database of ~2000 candidate images with a top-1 accuracy >70 times higher than a random baseline. Additionally, the bioimage encoder demonstrates remarkable transferability to various further prediction tasks within the domain of drug discovery, such as activity prediction, molecule classification, and mechanism of action identification. Thus, our approach not only addresses the current limitations of bioimaging databases but also paves the way towards foundation models for microscopy images.

Indexed as

Artificial IntelligenceLearningDatabases, FactualDrug DiscoveryKnowledge

Identifiers

PMID37957207
PMCPMC10643690

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.