ArticlePhilosophical transactions. Series A, Mathematical, physical, and engineering sciences2024
GPT-4 passes the bar exam.
Article in Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 64 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
64 citing papers in PubMed.
- Performance of ChatGPT, Claude, and AMBOSS on the European Board of Urology In-Service Assessment and Alignment With the European Association of Urology 2025 Guidelines: Comparative Study.JMIR formative research · 2026Article
- AI agents can coordinate via majority-following beyond human scale.Science advances · 2026Article
- Can artificial intelligence accurately assess systematic review quality? Benchmarking large language models for AMSTAR 2 appraisal in dental evidence synthesis.Evidence-based dentistry · 2026Article
- Evaluating and leveraging large language models in clinical pharmacology and therapeutics assessment: From exam takers to exam shapers.British journal of clinical pharmacology · 2026Article
- There is no free benchmark: An institutional view of legal AI benchmarking.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Deficient executive control in transformer attention.PNAS nexus · 2026Article
- Reimagining pharmacology education.Pharmacological reviews · 2026Review
- Teaching AI to handle exceptions: Supervised fine-tuning with human-aligned judgment.PNAS nexus · 2026Article
- Beyond human gold standards: A multimodel framework for automated abstract classification and information extraction.Research synthesis methods · 2026Article
- A rapid evidence review of evaluation techniques for large language models in legal use cases: trends, gaps, and recommendations for future research.AI & society · 2026Review
- Article
- OSCEai: personalized interactive learning for undergraduate medical education.Canadian medical education journal · 2025Article
- Stylistic language drives perceived moral superiority of LLMs.Scientific reports · 2025Article
- A personal health large language model for sleep and fitness coaching.Nature medicine · 2025Article
- ChatGPT's Performance on the Orthopaedic In-Training Examination (OITE): No Better Than a PGY3 Resident?Cureus · 2025Article
- Assessing the accuracy and readability of ChatGPT 4.0's original and simplified responses to common patient questions regarding periacetabular osteotomy.Journal of experimental orthopaedics · 2025Article
- Biostatisticians Meet AI: Navigating Shifts While Preserving Principles.Statistics in medicine · 2025Article
- A thematic analysis of what Australians state would change their minds on climate change.Scientific reports · 2025Article
- Extracting International Classification of Diseases Codes from Clinical Documentation Using Large Language Models.Applied clinical informatics · 2025Article
- How should the advancement of large language models affect the practice of science?Proceedings of the National Academy of Sciences of the United States of America · 2025Article
4 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
In this paper, we experimentally evaluate the zero-shot performance of GPT-4 against prior generations of GPT on the entire uniform bar examination (UBE), including not only the multiple-choice multistate bar examination (MBE), but also the open-ended multistate essay exam (MEE) and multistate performance test (MPT) components. On the MBE, GPT-4 significantly outperforms both human test-takers and prior models, demonstrating a 26% increase over ChatGPT and beating humans in five of seven subject areas. On the MEE and MPT, which have not previously been evaluated by scholars, GPT-4 scores an average of 4.2/6.0 when compared with much lower scores for ChatGPT. Graded across the UBE components, in the manner in which a human test-taker would be, GPT-4 scores approximately 297 points, significantly in excess of the passing threshold for all UBE jurisdictions. These findings document not just the rapid and remarkable advance of large language model performance generally, but also the potential for such models to support the delivery of legal services in society. This article is part of the theme issue 'A complexity science approach to law and governance'.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.