The Performance Ceiling of LIDC-IDRI Is Label-Defined Rather than Image-Defined: A Computable Bound and a Purposive Audit of Published Studies.

Benchmarks labelled by panels of human readers underpin most medical image analysis. We ask whether such a dataset has a single attainable ceiling. All 1319 annotation files of the Lung Image Database Consortium image collection (LIDC-IDRI) were parsed into 7034 reader-level ratings. Modelling each nodule by a latent malignancy propensity of which the four reader verdicts are conditionally independent draws, we prove that no image-based model can exceed, against a reader-derived label, the area under the receiver operating characteristic curve (AUC) of the propensity itself; we derive how that bound depends on the labelling policy, test the independence assumption against a prediction the fit does not use, simulate its failures, and audit a purposive sample of 22 published studies against the rules they declare. Among 905 nodules read by all four radiologists, all four agreed exactly for 11.7% and 58.0% carried ratings spanning two or more points. Across three latent families, retaining every nodule the ceiling is 0.898-0.913; discarding the 19.6% the readers split two-two lifts it to 0.946-0.954; retaining only unanimous nodules discards 57.7% and lifts it to 0.991-0.994. On the binary scale, no family predicted more reader agreement than observed; simulated reader heterogeneity would make these ceilings slightly low. Of 27 reported figures, 11 lie above the bound implied by the study's own declared rule, four beyond sampling error and two if modest heterogeneity is also allowed for. The ceiling is set by the label, not by the images.