Assessment of Inter-rater Variability in the Diagnosis of Urinary Tract Infections in the Emergency Department.

introductionUrinary tract infections (UTI) are among the most common bacterial infections diagnosed in the emergency department (ED), yet the urinalysis results can be neither sensitive nor specific for UTI. Our objective was to quantify inter-rater variability of three emergency attending physicians for the clinical diagnosis of UTI, and secondarily to compare the diagnosis made at bedside by the treating clinician with the evaluations of three emergency physician-chart reviewers after the fact.

methodsChart reviewers read 18 articles on the diagnosis of UTI before retrospectively evaluating a convenience sample of 473 ED encounters where patients received both a urinalysis and urine culture as part of their ED evaluation. The chart reviewers were blinded to the urine culture results, medications administered and prescribed, and to the treating clinician's diagnoses. Reviewers were asked to rate the likelihood of UTI based on a 0-4 ordinal scale. A "true positive" UTI occurred when the treating clinician diagnosed the patient with a UTI and the urine culture had ≥10,000 colony-forming units (CFU)/mL of bacteria. We considered a "false positive" to be when the treating clinician diagnosed the patient with a UTI, but the urine culture was < 10,000 CFU/mL of bacteria. A "true negative" occurred when the treating clinician did not diagnose the patient with a UTI, and the urine culture was < 10,000 CFU/mL of bacteria.

resultsMedian patient age was 63 years, 355 (75%) were female sex, 409 (86.5%) were White race, and 207 were admitted to the hospital. The inter-rater agreement among the three independent reviewers was high (κ 0.82-0.85) with intraclass coefficient (2,1) = 0.83. However, the reviewers-to-treating clinician agreement was only moderate in the true positives (treating clinician diagnosed patient with a UTI and the patient had a positive urine culture) and lowest in the false positives (treating clinician diagnosed the patient with a UTI, but the urine culture was < 10,000 CFU/mL with κ values of 0.44 and 0.21, respectively). The variables associated with consensus among reviewers were nitrites, leukocyte esterase, and higher urine white blood cells.

conclusionThere was high consensus among reviewers about the likelihood of a urinary tract infection, but lower consensus when comparing reviewers' impressions with those of the treating clinician. At bedside emergency clinicians were more likely to diagnose a UTI with a resultant negative urine culture. Further research is needed to improve the diagnostic accuracy of UTI in the emergency department.