Real-world diagnostic performance of artificial intelligence in skin cancer screening.

Background: The use of artificial intelligence (AI) in many aspects of society is expanding rapidly. AI has recently been employed in the National Health Service skin cancer referral pathway. We sought to assess the real-world diagnostic performance of AI in this setting. Objectives: To assess the real-world diagnostic performance, at the histopathological level, of AI employed in the urgent skin cancer screening pathway. Methods: This was a prospective observational study of the first 3 months of skin cancer referrals assessed by the Deep Ensemble for Recognition of Malignancy (DERM) AI algorithm in a tertiary care dermatology department in the North West of England. All lesions assessed by the algorithm were included in the analysis. Participant data were retrieved from medical records. Outcomes assessed included the AI diagnosis, whether a human review of AI diagnosis occurred, the face-to-face dermatologist's diagnosis and the outcome of the dermatologist's assessment. Comparison was made particularly between the final histopathological diagnosis, and AI and dermatologist diagnoses. Results: AI had a sensitivity of 95.3% [95% confidence interval (CI) 90.5-98.1], which compared favourably with dermatologists (88.5%, 95% CI 82.3-93.2; Conclusions: AI has high sensitivity in the detection of skin cancer. However, the diagnostic accuracy of the information provided by AI to clinicians is low and could be further optimized to reduce the risk of automation bias. Furthermore, this study suggests the removal of human validation of AI decisions may be premature due to the potential for missed cancer diagnoses.