ArticleJournal of cheminformatics2023
Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets.
Article in Journal of cheminformatics, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 13 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
13 citing papers in PubMed.
- Predictive performance precision analysis in medicine: identification of low-confidence predictions at patient and profile levels (MED3pa I).Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Uncertainty Quantification forChemical reviews · 2026Review
- Explicit Applicability Domain Calculations Can Help Determine When Uncertainty Estimates Are Less Reliable.ACS omega · 2026Article
- Uncertainty Quantification in Molecular Machine Learning for Property Predictions under Data Shifts.Journal of chemical information and modeling · 2026Article
- Generative Multiobjective Bayesian Optimization with Scalable Batch Evaluations for Sample-Efficient De Novo Molecular Design.Industrial & engineering chemistry research · 2026Article
- Leveraging uncertainty estimates for drug response prediction in cancer cell lines.Bioinformatics advances · 2026Article
- Combining Bayesian and Evidential Uncertainty Quantification for Improved Bioactivity Modeling.Journal of chemical information and modeling · 2025Article
- NOCTIS: open-source toolkit that turns reaction data into actionable graph networks.Journal of cheminformatics · 2025Article
- Data Generation for Machine Learning Interatomic Potentials and Beyond.Chemical reviews · 2024Review
- Uncertainty Quantification and Flagging of Unreliable Predictions in Predicting Mass Spectrometry-Related Properties of Small Molecules Using Machine Learning.International journal of molecular sciences · 2024Article
- UNIQUE: A Framework for Uncertainty Quantification Benchmarking.Journal of chemical information and modeling · 2024Article
- Spatially Resolved Uncertainties for Machine Learning Potentials.Journal of chemical information and modeling · 2024Article
- Reducing overconfident errors in molecular property classification using Posterior Network.Patterns (New York, N.Y.) · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
Abstract
With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman's rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. (Sensors 2022, 22, 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman's rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman's rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.