ArticleJournal of the American Medical Informatics Association : JAMIA2024
Can large language models provide secondary reliable opinion on treatment options for dermatological diseases?
Article in Journal of the American Medical Informatics Association : JAMIA, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
11 citing papers in PubMed, 1 synthesis or guideline pooled it, 18 citations in OpenAlex.
- Impact of large language model (ChatGPT) in healthcare: an umbrella review and evidence synthesis.Journal of biomedical science · 2025Pooled it
- Evaluating GPT-4 Responses on Scars or Keloids for Patient Education: Large Language Model Evaluation Study.JMIR medical informatics · 2026Article
- Evaluation of large language models in nutrition risk screening: a comparative analysis across 8 LLMs based on real-world EHR datasets.BMC medical informatics and decision making · 2026Article
- [Large language models-hype or hope?]Dermatologie (Heidelberg, Germany) · 2025Article
- ChatGPT Performance Deteriorated in Patients with Comorbidities When Providing Cardiological Therapeutic Consultations.Healthcare (Basel, Switzerland) · 2025Article
- The Role of ChatGPT in Dermatology Diagnostics.Diagnostics (Basel, Switzerland) · 2025Review
- Critical care studies using large language models based on electronic healthcare records: A technical note.Journal of intensive medicine · 2025Review
- Evaluating Large Language Models for Accuracy and Completeness of Vitiligo Patient Education: A Comparative Analysis.Clinical, cosmetic and investigational dermatology · 2025Article
- Navigating the potential and pitfalls of large language models in patient-centered medication guidance and self-decision support.Frontiers in medicine · 2025Article
- Generative artificial intelligence (AI): a key innovation or just hype in primary care settings?BMJ health & care informatics · 2024Article
- A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports.Journal of the American Medical Informatics Association : JAMIA · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 5 institutions in 4 countries.
Funding
Abstract
objectiveTo investigate the consistency and reliability of medication recommendations provided by ChatGPT for common dermatological conditions, highlighting the potential for ChatGPT to offer second opinions in patient treatment while also delineating possible limitations. MATERIALS AND
methodsIn this mixed-methods study, we used survey questions in April 2023 for drug recommendations generated by ChatGPT with data from secondary databases, that is, Taiwan's National Health Insurance Research Database and an US medical center database, and validated by dermatologists. The methodology included preprocessing queries, executing them multiple times, and evaluating ChatGPT responses against the databases and dermatologists. The ChatGPT-generated responses were analyzed statistically in a disease-drug matrix, considering disease-medication associations (Q-value) and expert evaluation.
resultsChatGPT achieved a high 98.87% dermatologist approval rate for common dermatological medication recommendations. We evaluated its drug suggestions using the Q-value, showing that human expert validation agreement surpassed Q-value cutoff-based agreement. Varying cutoff values for disease-medication associations, a cutoff of 3 achieved 95.14% accurate prescriptions, 5 yielded 85.42%, and 10 resulted in 72.92%. While ChatGPT offered accurate drug advice, it occasionally included incorrect ATC codes, leading to issues like incorrect drug use and type, nonexistent codes, repeated errors, and incomplete medication codes.
conclusionChatGPT provides medication recommendations as a second opinion in dermatology treatment, but its reliability and comprehensiveness need refinement for greater accuracy. In the future, integrating a medical domain-specific knowledge base for training and ongoing optimization will enhance the precision of ChatGPT's results.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.