ArticleUrology research & practice2023
Assessing the Performance of Chat Generative Pretrained Transformer (ChatGPT) in Answering Andrology-Related Questions.
Article in Urology research & practice, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The potential of large language models in the field of infertility: a systematic review.Journal of assisted reproduction and genetics · 2025Pooled it
- Utilization and prospects of artificial intelligence in the diagnosis, prediction, and treatment of erectile dysfunction.Sexual medicine · 2026Review
- From ChatGPT to DeepSeek: assessing the accuracy and readability of AI-generated erectile dysfunction responses.BMC urology · 2026Article
- Evaluation of DeepSeek-R1 and ChatGPT-4o as educational sources for upper tract urothelial carcinoma.Central European journal of urology · 2026Article
- Could ChatGPT Be a Tool Capable of Providing Qualified, Empathetic, and Assertive Answers to Patients After Bariatric Surgery? A Comparative Analysis of Its Versions.Obesity surgery · 2025Article
- ChatGPT Provides Accurate but Incomplete Responses and Reliably Adjusts Readability to Prompts for Hamstring Injury Frequently Asked Questions.Arthroscopy, sports medicine, and rehabilitation · 2025Article
- Artificial intelligence-generated responses to frequently asked questions on coccydynia: Evaluating the accuracy and consistency of GPT-4o's performance.Archives of rheumatology · 2025Article
- Assessing the performance of ChatGPT and Bard/Gemini against radiologists for Prostate Imaging-Reporting and Data System classification based on prostate multiparametric MRI text reports.The British journal of radiology · 2025Article
- Artificial Intelligence-Based Clinical Decision-Making in Erectile Dysfunction: a Narrative Review.Current urology reports · 2024Review
- Artificial Intelligence for Clinical Management of Male Infertility, a Scoping Review.Current urology reports · 2024Article
- ChatGPT as a Clinical Decision Maker for Urolithiasis: Compliance with the Current European Association of Urology Guidelines.European urology open science · 2024Review
- Response to commentary on: assessing ChatGPT's ability to answer questions pertaining to erectile dysfunction: can our patients trust it?International journal of impotence research · 2024Article
- Accuracy and Readability of Artificial Intelligence Chatbot Responses to Vasectomy-Related Questions: Public Beware.Cureus · 2024Article
- Performance of ChatGPT in providing patient information about upper tract urothelial carcinoma.Contemporary oncology (Poznan, Poland) · 2024Article
- Review
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveThe internet and social media have become primary sources of health information, with men frequently turning to these platforms before seeking professional help. Chat generative pretrained transformer (ChatGPT), an artificial intelligence model developed by OpenAI, has gained popularity as a natural language processing program. The present study evaluated the accuracy and reproducibility of ChatGPT's responses to andrology-related questions.
methodsThe study analyzed frequently asked andrology questions from health forums, hospital websites, and social media platforms like YouTube and Instagram. Questions were categorized into topics like male hypogonadism, erectile dysfunction, etc. The European Association of Urology (EAU) guideline recommendations were also included. These questions were input into ChatGPT, and responses were evaluated by 3 experienced urologists who scored them on a scale of 1 to 4.
resultsOut of 136 evaluated questions, 108 met the criteria. Of these, 87.9% received correct and adequate answers, 9.3% were correct but insufficient, and 3 responses contained both correct and incorrect information. No question was answered completely wrong. The highest correct answer rates were for disorders of ejaculation, penile curvature, and male hypogonadism. The EAU guideline-based questions achieved a correctness rate of 86.3%. The reproducibility of the answers was over 90%.
conclusionThe study found that ChatGPT provided accurate and reliable answers to over 80% of andrology-related questions. While limitations exist, such as potential outdated data and inability to understand emotional aspects, ChatGPT's potential in the health-care sector is promising. Collaborating with health-care professionals during artificial intelligence model development could enhance its reliability.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.