ArticleBMC medical research methodology2026
Compliance of systematic reviews and meta-analyses in ophthalmology with the PRISMA statement: an AI-based assessment and longitudinal comparison with 2017 data.
Article in BMC medical research methodology, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundSystematic reviews and meta-analyses are vital in evidence-based medicine, especially in ophthalmology, where the complexity of paired data can lead to reporting challenges. In 2017, we evaluated the adherence of ophthalmology-related systematic reviews and meta-analyses to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2009 statement. This study revisits the issue with a focus on adherence to the updated PRISMA 2020 checklist, compares results with the 2017 study, and explores the potential of AI in evaluating compliance.
objectiveThe aim of this study is to evaluate the reporting quality of systematic reviews and meta-analyses published in major ophthalmology journals between 2020 and 2024, based on the PRISMA 2020 checklist, and to compare human and AI assessments of compliance.
methodsA total of 207 systematic reviews and meta-analyses published in 11 major ophthalmology journals were included in this study. Each article was independently assessed for adherence to the 2020 PRISMA checklist, first by two human reviewers, and subsequently by two distinct AI platforms (ChatGPT-4.0 and Gemini Pro 2.5). Compliance scores were calculated, and inter-observer agreement between human and AI evaluations was determined using Cohen’s kappa statistic. The Mann–Whitney U test was employed to compare these findings with those of a 2017 study.
resultsThe mean compliance score, as assessed by human reviewers, was 36.28 out of 42 points (86.37%), indicating a substantial improvement in adherence to the PRISMA checklist compared with the level reported in the 2017 study (p < 0.00001). Compliance scores generated by the AI platforms demonstrated a moderate level of agreement with human assessments (Cohen’s κ = 0.63 for ChatGPT, 0.54 for Gemini). Strong compliance was observed for background and rationale (items 3 and 4), selection criteria (items 5–10b), and limitations (items 23a–23c). Conversely, lower compliance was noted for risk of bias assessment (item 11), sensitivity analysis (items 13f and 20c), and research registration (items 24a–24c).
conclusionsThis study demonstrates a marked improvement in the reporting quality of systematic reviews and meta-analyses in ophthalmology following adoption of the 2020 PRISMA statement. Nonetheless, persistent deficiencies remain, particularly in the reporting of bias, sensitivity analyses, and research registration. The application of AI models offers promising potential for enhancing the efficiency and effectiveness of reporting quality assessments; however, further refinement is required to ensure consistency and accuracy. Future iterations of the PRISMA guidelines should consider explicitly addressing the role of AI in research evaluation.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.