ArticleEuropean archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery2024
Validation of the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platforms.
Article in European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 45 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
45 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Current role of artificial intelligence and machine learning: is their application feasible in pediatric upper airway obstructive disorders?European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Pooled it
- AI for IMPACTS Framework for Evaluating the Long-Term Real-World Impacts of AI-Powered Clinician Tools: Systematic Review and Narrative Synthesis.Journal of medical Internet research · 2025Pooled it
- Benchmarking Generative Artificial Intelligence for the Analysis of Racial and Gender Disparities in Academic Ophthalmology.Journal of racial and ethnic health disparities · 2026Article
- ChatGPT Health and ChatGPT Plus in Urogynecology: A Blinded Comparison of Response Quality, Guideline Concordance, and Safety.International urogynecology journal · 2026Article
- Artificial Intelligence for Diagnosing Normal Anatomical Variants and Pathological Oral Mucosal Lesions: A Prospective Observational Study.Medicina (Kaunas, Lithuania) · 2026Observational
- Improving Pelvic Floor Disorder Education: A Second Pilot of a Retrieval-Augmented AI Chatbot Model.International urogynecology journal · 2026Article
- Leveraging artificial intelligence to streamline documentation and support patient-centered gynecologic oncology outpatient visits.Gynecologic oncology reports · 2026Article
- The Association of Micro- and Macro-Nutrient Patterns With the Risk of Rheumatoid Arthritis in Newly Diagnosed Cases: A Case-Control Study.Health science reports · 2026Article
- Dr. AI says you have sleep apnea: measuring the accuracy of patient-facing AI in sleep medicine.Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine · 2026Article
- "MELMA" in otolaryngology: Medical evaluation of large language model answers. Clinician-rated scoring (MELMA-Q) and web-based auditing (MELMA-W) novel tools for AI assessment.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Article
- Optimizing a Multimodal Large Language Model for Ultrasound-Based Thyroid Nodule Malignancy Classification: A Comparative Study of Few-Shot Learning, Prompt Engineering, and Fine-Tuning.Diagnostics (Basel, Switzerland) · 2026Article
- Development and Evaluation of an Augmented Artificial Intelligence Model for Urogynecology Queries.International urogynecology journal · 2026Article
- Generative AI vs web search for patient education: a comparative evaluation of OSA information quality.Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine · 2026Article
- Retrieval-Augmented Generation Versus GPT-4o for Patient-Facing Gynecological Cancer Information: Quality Evaluation.JMIR formative research · 2026Article
- Can Large Artificial Intelligence-Based Linguistic Models Help to Obtain Information About Burning Mouth Syndrome?Oral diseases · 2026Article
- A comparison of large language model-generated and published perioperative neurocognitive disorder recommendations: a cross-sectional web-based analysis.British journal of anaesthesia · 2026Article
- ChatGPT models provide higher-quality but lower-readability responses than Google Gemini regarding anterior shoulder instability, with no added benefit of the orthopaedic expert plugin.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Article
- Comparative analysis of six large language models in perioperative decision support for geriatric patients with multimorbidity: a three-dimensional evaluation framework.BMC anesthesiology · 2026Article
- Evaluation of ChatGPT-5.2 responses to frequently asked questions about benign bone tumors.Frontiers in oncology · 2026Article
- Readability and Quality of Chatbot Responses to Periodontal Patient Queries: A Cross-Sectional Evaluation of Three Publicly Accessible Large Language Models.International journal of dentistry · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
32 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThe widespread diffusion of Artificial Intelligence (AI) platforms is revolutionizing how health-related information is disseminated, thereby highlighting the need for tools to evaluate the quality of such information. This study aimed to propose and validate the Quality Assessment of Medical Artificial Intelligence (QAMAI), a tool specifically designed to assess the quality of health information provided by AI platforms.
methodsThe QAMAI tool has been developed by a panel of experts following guidelines for the development of new questionnaires. A total of 30 responses from ChatGPT4, addressing patient queries, theoretical questions, and clinical head and neck surgery scenarios were assessed by 27 reviewers from 25 academic centers worldwide. Construct validity, internal consistency, inter-rater and test-retest reliability were assessed to validate the tool.
resultsThe validation was conducted on the basis of 792 assessments for the 30 responses given by ChatGPT4. The results of the exploratory factor analysis revealed a unidimensional structure of the QAMAI with a single factor comprising all the items that explained 51.1% of the variance with factor loadings ranging from 0.449 to 0.856. Overall internal consistency was high (Cronbach's alpha = 0.837). The Interclass Correlation Coefficient was 0.983 (95% CI 0.973-0.991; F (29,542) = 68.3; p < 0.001), indicating excellent reliability. Test-retest reliability analysis revealed a moderate-to-strong correlation with a Pearson's coefficient of 0.876 (95% CI 0.859-0.891; p < 0.001).
conclusionsThe QAMAI tool demonstrated significant reliability and validity in assessing the quality of health information provided by AI platforms. Such a tool might become particularly important/useful for physicians as patients increasingly seek medical information on AI platforms.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.