ArticleScientific reports2025
Using generative AI for the objective assessment of language in healthcare.
Article in Scientific reports, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Traditional methods for language assessment in psychiatric and neurological disorders, such as clinical scales, are time and resource intensive, and can be hampered by rater biases and subjectivity. These limitations can compromise their reliability and sensitivity, as well as their practical use to measure change over time, which is of particular importance in clinical trials. Objective methods are required to improve the evaluation of language function across a spectrum of psychiatric and neurological conditions. To address these challenges, we introduce an innovative method that uses an artificial intelligence (AI) model, GPT-4, to provide an objective evaluation of language. As a test case, we focus on measuring expressive communication capabilities in autistic participants as they naturally converse with their study partner during an observational clinical trial. The conversations were recorded, professionally transcribed, and then processed with GPT-4 with the aim of predicting the individuals' Vineland Adaptive Behaviour Scales (VABS-II) expressive communication scores. The model's predictions were also compared with several benchmark linguistic features (e.g. the number of words spoken per sentence), to determine the added benefit of using such a complex model. We found that GPT-4's predictions correlated strongly with the actual VABS-II scores (Pearson's r > 0.65) and demonstrated high test-retest reliability (ICC(2,1) = 0.97). The model's predictions also accounted for significantly more variance than that explained by the benchmark linguistic features. These findings demonstrate that GPT-4 can provide a holistic, reliable, objective, and time-efficient assessment of expressive communication abilities. This suggests that generative AI models like GPT-4 could transform the assessment of communicative abilities, to support the assessment of treatment efficacy in clinical trials, and provide a faster and more scalable tool for assessing patients in clinical practice.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.