ArticleJMIR medical informatics2022
Utility Metrics for Evaluating Synthetic Health Data Generation Methods: Validation Study.
Article in JMIR medical informatics, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 25 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
25 citing papers in PubMed.
- Anonymized but Useful Synthetic Tabular Health Data for AI based Fall Risk Assessment.Scientific data · 2026Article
- Evaluating the quality of tabular synthetic data in health care.PLOS digital health · 2026Article
- Real-time prediction of trauma-induced coagulopathy using an inverted transformer (trauma-former): a methodological feasibility and simulation study based on the ADEMP framework.BMC medical research methodology · 2026Article
- An ELIXIR scoping review on domain-specific evaluation metrics for synthetic data in life sciences.NAR genomics and bioinformatics · 2026Article
- A call for principle-based acceptance of synthetic patients: establishing the AI-driven foundation for regulatory submissions.Frontiers in digital health · 2026Review
- Should we synthesize more than we need: impact of synthetic data generation for high-dimensional cross-sectional medical data.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Magnitude and Impact of Hallucinations in Tabular Synthetic Health Data on Prognostic Machine Learning Models: Validation Study.Journal of medical Internet research · 2025Article
- A review on generative AI models for synthetic medical text, time series, and longitudinal data.NPJ digital medicine · 2025Article
- Augmenting Insufficiently Accruing Oncology Clinical Trials Using Generative Models: Validation Study.Journal of medical Internet research · 2025Article
- How good is your synthetic data? SynthRO, a dashboard to evaluate and benchmark synthetic tabular data.BMC medical informatics and decision making · 2025Article
- Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees.Frontiers in digital health · 2025Article
- Gait stability prediction through synthetic time-series and vision-based data.Frontiers in sports and active living · 2025Article
- A synthetic data-driven machine learning approach for athlete performance attenuation prediction.Frontiers in sports and active living · 2025Article
- Large language models generating synthetic clinical datasets: a feasibility and comparative analysis with real-world perioperative data.Frontiers in artificial intelligence · 2025Article
- An assessment of synthetic data generation, use and disclosure under Canadian privacy regulations.AI and ethics · 2025Article
- Review
- Synthesis and quality assessment of combined time-series and static medical data using a real-world time-series generative adversarial network.Scientific reports · 2024Article
- An evaluation of the replicability of analyses using synthetic health data.Scientific reports · 2024Article
- Entropy removal of medical diagnostics.Scientific reports · 2024Article
- Introducing SPINE: A Holistic Approach to Synthetic Pulmonary Imaging Evaluation Through End-to-End Data and Model Management.IEEE open journal of engineering in medicine and biology · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundA regular task by developers and users of synthetic data generation (SDG) methods is to evaluate and compare the utility of these methods. Multiple utility metrics have been proposed and used to evaluate synthetic data. However, they have not been validated in general or for comparing SDG methods.
objectiveThis study evaluates the ability of common utility metrics to rank SDG methods according to performance on a specific analytic workload. The workload of interest is the use of synthetic data for logistic regression prediction models, which is a very frequent workload in health research.
methodsWe evaluated 6 utility metrics on 30 different health data sets and 3 different SDG methods (a Bayesian network, a Generative Adversarial Network, and sequential tree synthesis). These metrics were computed by averaging across 20 synthetic data sets from the same generative model. The metrics were then tested on their ability to rank the SDG methods based on prediction performance. Prediction performance was defined as the difference between each of the area under the receiver operating characteristic curve and area under the precision-recall curve values on synthetic data logistic regression prediction models versus real data models.
resultsThe utility metric best able to rank SDG methods was the multivariate Hellinger distance based on a Gaussian copula representation of real and synthetic joint distributions.
conclusionsThis study has validated a generative model utility metric, the multivariate Hellinger distance, which can be used to reliably rank competing SDG methods on the same data set. The Hellinger distance metric can be used to evaluate and compare alternate SDG methods.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.