ArticleThe open nursing journal2017
High Agreement and High Prevalence: The Paradox of Cohen's Kappa.
Article in The open nursing journal, 2017. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 73 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
73 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- A Systematic Review of Available Normative Data on Neuropsychological Tests for Spanish Speakers in the U.S., Latin America and the Caribbean, and Spain.Neuropsychology review · 2026Pooled it
- Social connectedness and disconnectedness in individuals with recent onset psychosis and suicidal experiences: a systematic review of the evidence.BMC psychiatry · 2026Pooled it
- Teaching clinical communication skills through virtual patient-based learning: an umbrella review of systematic reviews.BMC medical education · 2026Pooled it
- Individual-level agreement between coverage surveys and treatment registers in mass drug administration: A secondary analysis of the multi-country DeWorm3 cluster randomized trial.PLoS neglected tropical diseases · 2026Trial
- Brain Atrophy and White Matter Changes Grading Agreement on NCCT and MRI in Ischemic Stroke.Clinical neuroradiology · 2026Trial
- Testing the Evaluation of the Testimony of the Global Evaluation System in Bullying Cases.Psychosocial intervention · 2026Trial
- Sensitivity of Ocular Trauma Score (OTS) classification to missing and undocumented prognostic variables: A robustness analysis in open globe injury.Eye (London, England) · 2026Article
- Development and reliability testing of BRANT-MERQS-2A: A quality assessment tool for medication review type 2a reports.International journal of clinical pharmacy · 2026Article
- Common and Differing Elements Across Caregiver-Mediated Social Communication Programs Supporting Autistic Children: A Scoping Review With Content Analysis.Autism : the international journal of research and practice · 2026Article
- A medically grounded LLM agent-based tool to detect patient safety events in medical records.PLOS digital health · 2026Article
- Retrospective evaluation of the mr#ENZIAN classification's applicability in the pre-operative assessment of complex endometriosis: a compartment-dependent challenge.Abdominal radiology (New York) · 2026Article
- Performance of Two AI Approaches in ASReview Compared With Manual Screening for Dementia Care Literature Screening: Comparative Analysis.JMIR formative research · 2026Article
- Iterative Multidisciplinary Development and Evaluation of a Patient-Facing Social Determinants of Health Chatbot Using Synthetic Data Simulation: Mixed Methods Study.JMIR formative research · 2026Article
- The Therapeutic Landscape of Nausea and Vomiting of Pregnancy and Hyperemesis Gravidarum: Five Decades of Evolving Treatment and Supportive Care.Medical sciences (Basel, Switzerland) · 2026Review
- Reproducibility of an 85-item Child Food Frequency Questionnaire (85-item Child-FFQ) and diet quality scores among Polish children: The cross-sectional PITNUTS 2024 study.BMC pediatrics · 2026Observational
- Uneven integration of health in China's territorial spatial planning 2035.BMC public health · 2026Article
- Comparative Evaluation of Real-Time PCR, Immunochromatographic Assay, and Modified Carbapenem Inactivation Method for Carbapenemase Detection in Enterobacterales Isolates.International journal of molecular sciences · 2026Article
- Clinical Evaluation of ChatGPT-5.3 Responses to Patient-Oriented Questions on Scoliosis: A Multidimensional Expert Analysis.Healthcare (Basel, Switzerland) · 2026Article
- What's in a Prompt? Assignment Prompt Cues Influence Data Analysis, Critique, and Inclusion of Follow-up Experiments.CBE life sciences education · 2026Article
- Expert Evaluation and Consensus on GPT-4o Summaries of Clinical Letters: Validation and Results of the Framework and Implementation of AI Tools Project.JMIR medical informatics · 2026Article
13 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundCohen's Kappa is the most used agreement statistic in literature. However, under certain conditions, it is affected by a paradox which returns biased estimates of the statistic itself.
objectiveThe aim of the study is to provide sufficient information which allows the reader to make an informed choice of the correct agreement measure, by underlining some optimal properties of Gwet's AC1 in comparison to Cohen's Kappa, using a real data example.
methodDuring the process of literature review, we have asked a panel of three evaluators to come up with a judgment on the quality of 57 randomized controlled trials assigning a score to each trial using the Jadad scale. The quality was evaluated according to the following dimensions: adopted design, randomization unit, type of primary endpoint. With respect to each of the above described features, the agreement between the three evaluators has been calculated using Cohen's Kappa statistic and Gwet's AC1 statistic and, finally, the values have been compared with the observed agreement.
resultsThe values of the Cohen's Kappa statistic would lead to believe that the agreement levels for the variables Unit, Design and Primary Endpoints are totally unsatisfactory. The AC1 statistic, on the contrary, shows plausible values which are in line with the respective values of the observed concordance.
conclusionWe conclude that it would always be appropriate to adopt the AC1 statistic, thus bypassing any risk of incurring the paradox and drawing wrong conclusions about the results of agreement analysis.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.