ArticlePloS one2014
Using ROC curves to choose minimally important change thresholds when sensitivity and specificity are valued equally: the forgotten lesson of pythagoras. theoretical considerations and an example application of change in health status.
Article in PloS one, 2014. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 64 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
64 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Pearls and pitfalls of PROMIS clinically significant outcomes in orthopaedic surgery.Archives of orthopaedic and trauma surgery · 2023Pooled it
- Optimising plasma clozapine levels to improve treatment response: an individual patient data meta-analysis and receiver operating characteristic curve analysis.The British journal of psychiatry : the journal of mental science · 2023Pooled it
- Pooled it
- Estimation of Minimal Clinically Important Difference for Tinnitus Handicap Inventory and Tinnitus Functional Index.Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery · 2025Trial
- Psychometric validation of the severity of chronic cough diary, leicester cough questionnaire, and a cough severity visual analogue scale in patients with refractory chronic cough.Journal of patient-reported outcomes · 2025Trial
- Meaningful change threshold estimation for the non-small cell lung cancer symptom assessment questionnaire (NSCLC-SAQ): psychometric analysis from a phase 3 trial (LIBRETTO-431).Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2025Trial
- Validity, reliability, responsiveness, and clinically meaningful change threshold estimates of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Breast Cancer Symptom Index (NFBSI-16).Journal of patient-reported outcomes · 2024Trial
- Psychometric Evaluation of the MENQOL Instrument in Women Experiencing Vasomotor Symptoms Associated with Menopause.Advances in therapy · 2024Trial
- Minimal important difference and patient acceptable symptom state for common outcome instruments in patients with a closed humeral shaft fracture - analysis of the FISH randomised clinical trial data.BMC medical research methodology · 2022Trial
- Minimal important difference and patient acceptable symptom state for pain, Constant-Murley score and Simple Shoulder Test in patients with subacromial pain syndrome.BMC medical research methodology · 2021Trial
- Estimation of minimally important differences and responder definitions for EORTC QLQ-MY20 scores in multiple myeloma patients.European journal of haematology · 2019Trial
- Assessing the Significance of Individual Change in 2 Samples of Patients in Treatment for Low Back Pain Using 5 Different Statistical Indicators.Journal of manipulative and physiological therapeuticsTrial
- Defining a Clinically Meaningful Within-Person Change Threshold for the Psychomotor Vigilance Test in Narcolepsy Type 1.Therapeutic innovation & regulatory science · 2026Article
- Key factors of the deranged antiviral response in elderly patients with COVID-19: a machine-learning analysis.GeroScience · 2026Article
- Severity strata and minimally important differences of CompAQ and Skindex-16 for patients with acne.Journal of the European Academy of Dermatology and Venereology : JEADV · 2026Article
- The diagnostic accuracy of the faecal immunochemical test for the detection of early-onset colorectal cancer: an age-stratified analysis in South West England.British journal of cancer · 2025Article
- Clinical value of biomarkers in relation to artery size in eosinophilic granulomatosis with polyangiitis: findings from an inception cohort at a Japanese City Hospital.Clinical rheumatology · 2025Article
- Clinical cut-offs for hip- and knee arthroplasty outcome - minimal clinically important improvement (MCII) and patient acceptable symptom state (PASS) of patient-reported outcome measures (PROM).Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2025Article
- Can patient self-evaluation of functional status be used for evaluation of impairment of motor function in Guillain-Barré syndrome? Mapping clinician- and patient-reported outcomes in a phase 3 study of eculizumab in Japan.Frontiers in neurology · 2025Article
- Construct validity, responsiveness and minimal important difference of the cumulated ambulation score in older adults with hip fracture in sub-acute rehabilitation facility.Clinical rehabilitation · 2024Observational
4 more citing papers are in PubMed but not listed here.
Corrections and comments
- Erratum issued
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundReceiver Operator Characteristic (ROC) curves are being used to identify Minimally Important Change (MIC) thresholds on scales that measure a change in health status. In quasi-continuous patient reported outcome measures, such as those that measure changes in chronic diseases with variable clinical trajectories, sensitivity and specificity are often valued equally. Notwithstanding methodologists agreeing that these should be valued equally, different approaches have been taken to estimating MIC thresholds using ROC curves. AIMS AND
objectivesWe aimed to compare the different approaches used with a new approach, exploring the extent to which the methods choose different thresholds, and considering the effect of differences on conclusions in responder analyses.
methodsUsing graphical methods, hypothetical data, and data from a large randomised controlled trial of manual therapy for low back pain, we compared two existing approaches with a new approach that is based on the addition of the sums of squares of 1-sensitivity and 1-specificity.
resultsThere can be divergence in the thresholds chosen by different estimators. The cut-point selected by different estimators is dependent on the relationship between the cut-points in ROC space and the different contours described by the estimators. In particular, asymmetry and the number of possible cut-points affects threshold selection.
conclusionChoice of MIC estimator is important. Different methods for choosing cut-points can lead to materially different MIC thresholds and thus affect results of responder analyses and trial conclusions. An estimator based on the smallest sum of squares of 1-sensitivity and 1-specificity is preferable when sensitivity and specificity are valued equally. Unlike other methods currently in use, the cut-point chosen by the sum of squares method always and efficiently chooses the cut-point closest to the top-left corner of ROC space, regardless of the shape of the ROC curve.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.