ArticleStatistical methods in medical research
Asymptotic variability of (multilevel) multirater kappa coefficients.
Article in Statistical methods in medical research. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Deployability of machine-learning prediction models in elite and high-level team sports: a task-specific systematic review.Frontiers in physiology · 2026Pooled it
- High interobserver variability exists in grading appendiceal goblet cell adenocarcinoma using World Health Organization 5th edition criteria.Histopathology · 2026Article
- Reliability of Physical Examination Findings in Youths Diagnosed With Pneumonia.JAMA network open · 2026Article
- Measuring Agreement in Diagnostics: A Practical Guide for Researchers.Statistics in medicine · 2025Article
- Artificial Intelligence-Based Detection of Central Retinal Artery Occlusion Within 4.5 Hours on Standard Fundus Photographs.Journal of the American Heart Association · 2025Observational
- Deep Learning to Discriminate Arteritic From Nonarteritic Ischemic Optic Neuropathy on Color Images.JAMA ophthalmology · 2024Article
- Interobserver agreement on line-field confocal optical coherence tomography image markers in keratinocyte carcinomas and precursor lesions.Archives of dermatological research · 2024Article
- Comparison ofCurrent oncology (Toronto, Ont.) · 2024Article
- Development and Validation of a Remote Monitoring Tool for Assessment of Mild, Moderate, and Severe Infections in Inflammatory Bowel Disease.Journal of Crohn's & colitis · 2023Article
- Analysis of the Weighted Kappa and Its Maximum with Markov Moves.Psychometrika · 2022Article
- Robustness ofStatistics in medicine · 2022Article
- Comparison of Hydrocolloid Dressings and Silver Nanoparticles in Treatment of Pressure Ulcers in Patients with Spinal Cord Injuries: A Randomized Clinical Trial.Journal of caring sciences · 2022Article
- Accuracy of a Deep Learning System for Classification of Papilledema Severity on Ocular Fundus Photographs.Neurology · 2021Article
- Prevalence and clinical associations of wheezes and crackles in the general population: the Tromsø study.BMC pulmonary medicine · 2019Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Agreement studies are of paramount importance in various scientific domains. When several observers classify objects on categorical scales, agreement can be quantified through multirater kappa coefficients. In most statistical packages, the standard error of these coefficients is only available under the null hypothesis that the coefficient is equal to zero, preventing the construction of confidence intervals in the general case. The aim of this paper is triple. First, simple analytic formulae for the standard error of multirater kappa coefficients will be given in the general case. Second, these formulae will be extended to the case of multilevel data structures. The formulae are based on simple matrix algebra and are implemented in the R package "multiagree". Third, guidelines on the choice between the different mulitrater kappa coefficients will be provided.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.