ArticlePloS one2019
Using natural language processing and machine learning to classify health literacy from secure messages: The ECLIPPSE study.
Article in PloS one, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Computational Use of Patient-Provider Secure Messaging Data to Achieve Better Clinical Efficiency and Quality of Communication: A Systematic Review.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2024Pooled it
- Challenges and Opportunities for Professional Medical Publications Writers to Contribute to Plain Language Summaries (PLS) in an AI/ML Environment - A Consumer Health Informatics Systematic Review.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2023Pooled it
- A scoping review of studies on secure messaging through patient portals: persistent challenges and potential solutions.npj health systems · 2026Article
- Development of Machine Learning Algorithms for Identifying Patients With Limited Health Literacy.Journal of evaluation in clinical practice · 2025Article
- Is the patient speaking or the nurse? Automatic speaker type identification in patient-nurse audio recordings.Journal of the American Medical Informatics Association : JAMIA · 2023Article
- Automated Strategy Feedback Can Improve the Readability of Physicians' Electronic Communications to Simulated Patients.International journal of human-computer studies · 2023Article
- Engaging Men of Diverse Racial and Ethnic Groups With Advanced Prostate Cancer in the Design of an mHealth Diet and Exercise Intervention: Focus Group Study.JMIR cancer · 2023Article
- Precision communication: Physicians' linguistic adaptation to patients' health literacy.Science advances · 2021Article
- Predicting the Easiness and Complexity of English Health Materials for International Tertiary Students With Linguistically Enhanced Machine Learning Algorithms: Development and Validation Study.JMIR medical informatics · 2021Article
- Developing and Testing Automatic Models of Patient Communicative Health Literacy Using Linguistic Features: Findings from the ECLIPPSE study.Health communication · 2021Article
- Use of Machine Learning Algorithms to Predict the Understandability of Health Education Materials: Development and Evaluation Study.JMIR medical informatics · 2021Article
- Validity of a Computational Linguistics-Derived Automated Health Literacy Measure Across Race/Ethnicity: Findings from The ECLIPPSE Project.Journal of health care for the poor and underserved · 2021Article
- Employing computational linguistics techniques to identify limited patient health literacy: Findings from the ECLIPPSE study.Health services research · 2021Article
- Challenges and solutions to employing natural language processing and machine learning to measure patients' health literacy and physician writing complexity: The ECLIPPSE study.Journal of biomedical informatics · 2021Article
- A Direction for the Next Decade of NIH-Funded Health Literacy Research.Health literacy research and practice · 2020Article
- Use of a Machine Learning Program to Correctly Triage Incoming Text Messaging Replies From a Cardiovascular Text-Based Secondary Prevention Program: Feasibility Study.JMIR mHealth and uHealth · 2020Article
- The evolving landscape of artificial intelligence in patient education: A bibliometric knowledge mapping study.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
Abstract
Limited health literacy is a barrier to optimal healthcare delivery and outcomes. Current measures requiring patients to self-report limitations are time-consuming and may be considered intrusive by some. This makes widespread classification of patient health literacy challenging. The objective of this study was to develop and validate "literacy profiles" as automated indicators of patients' health literacy to facilitate a non-intrusive, economic and more comprehensive characterization of health literacy among a health care delivery system's membership. To this end, three literacy profiles were generated based on natural language processing (combining computational linguistics and machine learning) using a sample of 283,216 secure messages sent from 6,941 patients to their primary care physicians. All patients were participants in Kaiser Permanente Northern California's DISTANCE Study. Performance of the three literacy profiles were compared against a gold standard of patient self-reported health literacy. Associations were analyzed between each literacy profile and patient demographics, health outcomes and healthcare utilization. T-tests were used for numeric data such as A1C, Charlson comorbidity index and healthcare utilization rates, and chi-square tests for categorical data such as sex, race, poor adherence and severe hypoglycemia. Literacy profiles varied in their test characteristics, with C-statistics ranging from 0.61-0.74. Relations between literacy profiles and health outcomes revealed patterns consistent with previous health literacy research: patients identified via literacy profiles indicative of limited health literacy: (a) were older and more likely of minority status; (b) had poorer medication adherence and glycemic control; and (c) exhibited higher rates of hypoglycemia, comorbidities and healthcare utilization. This represents the first successful attempt to employ natural language processing to estimate health literacy. Literacy profiles can offer an automated and economical way to identify patients with limited health literacy and greater vulnerability to poor health outcomes.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.