ArticleJMIR medical education2023
The Potential of GPT-4 as a Support Tool for Pharmacists: Analytical Study Using the Japanese National Examination for Pharmacists.
Article in JMIR medical education, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 24 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
24 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Performance of ChatGPT-3.5 and GPT-4 in national licensing examinations for medicine, pharmacy, dentistry, and nursing: a systematic review and meta-analysis.BMC medical education · 2024Pooled it
- Comparative performance of large language models and Drugs.com versus Lexicomp for antiseizure medication drug-drug interactions: A cross-sectional study with iterative prompting analysis.Exploratory research in clinical and social pharmacy · 2026Article
- Artificial intelligence chatbots versus dentists: a comparative knowledge assessment on traumatic dental injury management.BMC oral health · 2026Article
- Drug-drug interaction identification using large language models.medRxiv : the preprint server for health sciences · 2025Article
- Performance Evaluation of 18 Generative AI Models (ChatGPT, Gemini, Claude, and Perplexity) in 2024 Japanese Pharmacist Licensing Examination: Comparative Study.JMIR medical education · 2025Article
- Young Adult Perspectives on Artificial Intelligence-Based Medication Counseling in China: Discrete Choice Experiment.Journal of medical Internet research · 2025Article
- Exploration of the optimal deep learning model for english-Japanese machine translation of medical device adverse event terminology.BMC medical informatics and decision making · 2025Article
- Integrating ChatGPT as a Tool in Pharmacy Practice: A Cross-Sectional Exploration Among Pharmacists in Saudi Arabia.Integrated pharmacy research & practice · 2025Article
- Large language models management of complex medication regimens: a case-based evaluation.Frontiers in pharmacology · 2025Article
- ChatGPT (GPT-4V) Performance on the Healthcare Information Technologist Examination in Japan.Cureus · 2025Article
- An exploratory assessment of GPT-4o and GPT-4 performance on the Japanese National Dental Examination.The Saudi dental journal · 2024Article
- Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis.Journal of healthcare informatics research · 2024Article
- Bioinformatics and biomedical informatics with ChatGPT: Year one review.Quantitative biology (Beijing, China) · 2024Article
- Evaluating the efficacy of leading large language models in the Japanese national dental hygienist examination: A comparative analysis of ChatGPT, Bard, and Bing Chat.Journal of dental sciences · 2024Article
- ChatGPT's Role in Improving Education Among Patients Seeking Emergency Medical Treatment.The western journal of emergency medicine · 2024Article
- Potential of ChatGPT to Pass the Japanese Medical and Healthcare Professional National Licenses: A Literature Review.Cureus · 2024Review
- Article
- Artificial intelligence generated clinical score sheets: looking at the two faces of Janus.Laboratory animal research · 2024Article
- Performance of GPT-4V in Answering the Japanese Otolaryngology Board Certification Examination Questions: Evaluation Study.JMIR medical education · 2024Article
- Utilizing ChatGPT in Telepharmacy.Cureus · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThe advancement of artificial intelligence (AI), as well as machine learning, has led to its application in various industries, including health care. AI chatbots, such as GPT-4, developed by OpenAI, have demonstrated potential in supporting health care professionals by providing medical information, answering examination questions, and assisting in medical education. However, the applicability of GPT-4 in the field of pharmacy remains unexplored.
objectiveThis study aimed to evaluate GPT-4's ability to answer questions from the Japanese National Examination for Pharmacists (JNEP) and assess its potential as a support tool for pharmacists in their daily practice.
methodsThe question texts and answer choices from the 107th and 108th JNEP, held in February 2022 and February 2023, were input into GPT-4. As GPT-4 cannot process diagrams, questions that included diagram interpretation were not analyzed and were initially given a score of 0. The correct answer rates were calculated and compared with the passing criteria of each examination to evaluate GPT-4's performance.
resultsFor the 107th and 108th JNEP, GPT-4 achieved an accuracy rate of 64.5% (222/344) and 62.9% (217/345), respectively, for all questions. When considering only the questions that GPT-4 could answer, the accuracy rates increased to 78.2% (222/284) and 75.3% (217/287), respectively. The accuracy rates tended to be lower for physics, chemistry, and calculation questions.
conclusionsAlthough GPT-4 demonstrated the potential to answer questions from the JNEP and support pharmacists' capabilities, it also showed limitations in handling highly specialized questions, calculation questions, and questions requiring diagram recognition. Further evaluation is necessary to explore its applicability in real-world clinical settings, considering the complexities of patient scenarios and collaboration with health care professionals. By addressing these limitations, GPT-4 could become a more reliable tool for pharmacists in their daily practice.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.