ArticlePloS one2019
Why Cohen's Kappa should be avoided as performance measure in classification.
Article in PloS one, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 55 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
55 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Pooled it
- Leveraging Machine Learning to Understand the Link Between School Climate and Youth Substance Use: a Focus on Cannabis and Alcohol Use.Prevention science : the official journal of the Society for Prevention Research · 2026Article
- Iterative Multidisciplinary Development and Evaluation of a Patient-Facing Social Determinants of Health Chatbot Using Synthetic Data Simulation: Mixed Methods Study.JMIR formative research · 2026Article
- DenseViT-OCT: A Hybrid CNN-Transformer Architecture with Multi-Scale Dense Feature Aggregation for Automated Epiretinal Membrane Severity Classification.Tomography (Ann Arbor, Mich.) · 2026Article
- Predicting identity dissociation using childhood maltreatment and genetic variation in the stress-response gene FKBP5: a machine learning analysis.Scientific reports · 2026Article
- A biologically structured hierarchical vision transformer-CNN framework for robust tomato leaf disease classification.Frontiers in artificial intelligence · 2026Article
- Assessment of transformer-based AI in clinical oncology.BMC medical informatics and decision making · 2025Article
- SATrans-Net: Sparse Attention Transformer for EEG-based motor imagery decoding.Scientific reports · 2025Article
- Assessing Changes in Grassland Species Distribution at the Landscape Scale Using Hyperspectral Remote Sensing.Sensors (Basel, Switzerland) · 2025Article
- Human-in-the-Loop Performance of LLM-Assisted Arterial Blood Gas Interpretation: A Single-Center Retrospective Study.Journal of clinical medicine · 2025Article
- The WHO BMI System Misclassifies Weight Status in Adults from the General Population in North Italy: A DXA-Based Assessment Study (18-98 Years).Nutrients · 2025Article
- Retinal optical coherence tomography angiography imaging in population studies for study of microvascular dysfunction in Alzheimer's disease and related dementias.Alzheimer's & dementia : the journal of the Alzheimer's Association · 2025Article
- Emotion-Aware RoBERTa enhanced with emotion-specific attention and TF-IDF gating for fine-grained emotion recognition.Scientific reports · 2025Article
- Predicting Leadership Status Through Trait Emotional Intelligence and Cognitive Ability.Behavioral sciences (Basel, Switzerland) · 2025Article
- Targeted syndromic next-generation sequencing panel for simultaneous detection of pathogens associated with bovine reproductive failure.Journal of clinical microbiology · 2025Article
- Predicting hypertension and identifying most important factors among married women in Bangladesh using machine learning approach.PloS one · 2025Article
- Prediction of Hearing Help Seeking to Design a Recommendation Module of an mHealth Hearing App: Intensive Longitudinal Study of Feature Importance Assessment.JMIR human factors · 2024Article
- NAPping PAnts (NAPPA): An open wearable solution for monitoring Infant's sleeping rhythms, respiration and posture.Heliyon · 2024Article
- Palms of the past: can morphometric phytolith analysis inform deep time evolution and palaeoecology of Arecaceae?Annals of botany · 2024Article
- Non-animal models for blood-brain barrier permeability evaluation of drug-like compounds.Scientific reports · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
We show that Cohen's Kappa and Matthews Correlation Coefficient (MCC), both extended and contrasted measures of performance in multi-class classification, are correlated in most situations, albeit can differ in others. Indeed, although in the symmetric case both match, we consider different unbalanced situations in which Kappa exhibits an undesired behaviour, i.e. a worse classifier gets higher Kappa score, differing qualitatively from that of MCC. The debate about the incoherence in the behaviour of Kappa revolves around the convenience, or not, of using a relative metric, which makes the interpretation of its values difficult. We extend these concerns by showing that its pitfalls can go even further. Through experimentation, we present a novel approach to this topic. We carry on a comprehensive study that identifies an scenario in which the contradictory behaviour among MCC and Kappa emerges. Specifically, we find out that when there is a decrease to zero of the entropy of the elements out of the diagonal of the confusion matrix associated to a classifier, the discrepancy between Kappa and MCC rise, pointing to an anomalous performance of the former. We believe that this finding disables Kappa to be used in general as a performance measure to compare classifiers.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.