ArticlePloS one2019
Scoring reading parameters: An inter-rater reliability study using the MNREAD chart.
Article in PloS one, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
11 citing papers in PubMed, 21 citations in OpenAlex.
- Functional acuity score as a surrogate of reading performance in low vision: A Japanese disability evaluation cohort study.PloS one · 2026Article
- Evaluating the Efficiency and Precision of Bayesian Active Learning qReading in Low Vision.Translational vision science & technology · 2025Article
- PTVR - A software in Python to make virtual reality experiments easier to build and more reproducible.Journal of vision · 2024Article
- Validation of a computerized version of the MNREAD chart for children.International ophthalmology · 2024Article
- Further Validation of Comfortable Print Size as a Parameter for Clinical Low-Vision Assessment.Translational vision science & technology · 2023Article
- The Basic VRS-Effect Study: Clinical Trial Outcomes and Cost-Effectiveness of Low Vision Rehabilitation in Portugal.Ophthalmology and therapy · 2023Article
- A systematic review of reading tests.International journal of ophthalmology · 2023Review
- Predictors of problems reported on the EQ-5D-3L dimensions among people with impaired vision in northern Portugal.Health and quality of life outcomes · 2022Observational
- Comparison of Reading Test Parameters from the Print and Tablet Application Forms of the Minnesota Low Vision Reading Test.Turkish journal of ophthalmology · 2022Article
- Introduction of a digital near-vision reading test for normal and low vision adults: development and validation.Eye and vision (London, England) · 2020Article
- Reading performance in Portuguese children from second to tenth grade with the MNREAD reading acuity test.Journal of optometryArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors at 6 institutions in 4 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
purposeFirst, to evaluate inter-rater reliability when human raters estimate the reading performance of visually impaired individuals using the MNREAD acuity chart. Second, to evaluate the agreement between computer-based scoring algorithms and compare them with human rating.
methodsReading performance was measured for 101 individuals with low vision, using the Portuguese version of the MNREAD test. Seven raters estimated the maximum reading speed (MRS) and critical print size (CPS) of each individual MNREAD curve. MRS and CPS were also calculated automatically for each curve using two different algorithms: the original standard deviation method (SDev) and a non-linear mixed effects (NLME) modeling. Intra-class correlation coefficients (ICC) were used to estimate absolute agreement between raters and/or algorithms.
resultsAbsolute agreement between raters was 'excellent' for MRS (ICC = 0.97; 95%CI [0.96, 0.98]) and 'moderate' to 'good' for CPS (ICC = 0.77; 95%CI [0.69, 0.83]). For CPS, inter-rater reliability was poorer among less experienced raters (ICC = 0.70; 95%CI [0.57, 0.80]) when compared to experienced ones (ICC = 0.82; 95%CI [0.76, 0.88]). Absolute agreement between the two algorithms was 'excellent' for MRS (ICC = 0.96; 95%CI [0.91, 0.98]). For CPS, the best possible agreement was found for CPS defined as the print size sustaining 80% of MRS (ICC = 0.77; 95%CI [0.68, 0.84]). Absolute agreement between raters and automated methods was 'excellent' for MRS (ICC = 0.96; 95% CI [0.88, 0.98] for SDev; ICC = 0.97; 95% CI [0.95, 0.98] for NLME). For CPS, absolute agreement between raters and SDev ranged from 'poor' to 'good' (ICC = 0.66; 95% CI [0.3, 0.80]), while agreement between raters and NLME was 'good' (ICC = 0.83; 95% CI [0.76, 0.88]).
conclusionFor MRS, inter-rater reliability is excellent, even considering the possibility of noisy and/or incomplete data collected in low-vision individuals. For CPS, inter-rater reliability is lower. This may be problematic, for instance in the context of multisite investigations or follow-up examinations. The NLME method showed better agreement with the raters than the SDev method for both reading parameters. Setting up consensual guidelines to deal with ambiguous curves may help improve reliability. While the exact definition of CPS should be chosen on a case-by-case basis depending on the clinician or researcher's motivations, evidence suggests that estimating CPS as the smallest print size sustaining about 80% of MRS would increase inter-rater reliability.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.