ArticleJournal of the Medical Library Association : JMLA2018
Computerized versus hand-scored health literacy tools: a comparison of Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid in printed patient education materials.
Article in Journal of the Medical Library Association : JMLA, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 45 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
45 citing papers in PubMed, 2 syntheses or guidelines pooled it, 94 citations in OpenAlex.
- Pooled it
- Addressing Health Literacy in Patient Decision Aids: An Update from the International Patient Decision Aid Standards.Medical decision making : an international journal of the Society for Medical Decision Making · 2021Pooled it
- Performance of large language models on the Turkish Pharmacy Specialty Examination: a comparative analysis of accuracy, confidence, and readability.Scientific reports · 2026Article
- When Large Language Models (LLMs) walk into a Bachelor's in optometry examination: Comparing the performances of LLMs and bachelor of optometry students.Indian journal of ophthalmology · 2026Article
- Article
- Comparative Assessment of Large Language Model Outputs and NHS Patient Information in Oral Medicine.Cureus · 2025Article
- Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study.Journal of medical Internet research · 2025Article
- Assessing the accuracy and explainability of using ChatGPT to evaluate the quality of health news.BMC public health · 2025Article
- Examining Health Literacy and Appropriateness of Commercially Developed Patient Education Materials for Adults Living With Diabetes on the U.S./Mexico Border.Clinical nursing research · 2025Article
- Readability of Patient Education Materials in Plastic Surgery: Assessing 14 Years of Progress.Plastic and reconstructive surgery. Global open · 2025Article
- Article
- Article
- Community Collaboration in Public Health Genetic Literacy: Methods for Co-Designing Educational Resources for Equitable Genomics Research and Practice.Public health genomics · 2025Article
- Evaluating Patient Literacy in the Health Information Space: What Does Release of Information Tell Us?Advances in health information science and practice · 2025Article
- Battle of the artificial intelligence: a comprehensive comparative analysis of DeepSeek and ChatGPT for urinary incontinence-related questions.Frontiers in public health · 2025Article
- Integrating large language models into clinical pharmacy education: applications in perioperative medication management for gastric cancer.Frontiers in medicine · 2025Article
- Reading Grade Level Estimates of Public Health Communication on Social Media Vary Due to Text Preprocessing.Journal of health communication · 2025Article
- Evaluation of the Quality and Readability of Web-Based Information Regarding Foreign Bodies of the Ear, Nose, and Throat: Qualitative Content Analysis.JMIR formative research · 2024Article
- Best practices on patient education materials in hip surgery based on learnings from major hip centers and societies.Journal of hip preservation surgery · 2024Article
- ChatGPT-3.5 and Bing Chat in ophthalmology: an updated evaluation of performance, readability, and informative sources.Eye (London, England) · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 1 institution in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveThe research compared and contrasted hand-scoring and computerized methods of evaluating the grade level of patient education materials that are distributed at an academic medical center in east Tennessee and sought to determine if these materials adhered to the American Medical Association's (AMA's) recommended reading level of sixth grade.
methodsLibrarians at an academic medical center located in the heart of Appalachian Tennessee initiated the assessment of 150 of the most used printed patient education materials. Based on the Flesch-Kincaid (F-K) scoring rubric, 2 of the 150 documents were excluded from statistical comparisons due to the absence of text (images only). Researchers assessed the remaining 148 documents using the hand-scored Simple Measure of Gobbledygook (SMOG) method and the computerized F-K grade level method. For SMOG, 3 independent reviewers hand-scored each of the 150 documents. For F-K, documents were analyzed using Microsoft Word. Reading grade levels scores were entered into a database for statistical analysis. Inter-rater reliability was calculated using intra-class correlation coefficients (ICC). Paired
resultsAcceptable inter-rater reliability was found for SMOG (ICC=0.95). For the 148 documents assessed, SMOG produced a significantly higher mean reading grade level (M=9.6, SD=1.3) than F-K (M=6.5, SD=1.3;
conclusionsComputerized health literacy assessment tools, used by many national patient education material providers, might not be representative of the actual reading grade levels of patient education materials. This is problematic in regions like Appalachia because materials may not be comprehensible to the area's low-literacy patients. Medical librarians have the potential to advance their role in patient education to better serve their patient populations.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.