Effectiveness evaluation of digital contact tracing for COVID-19 in New South Wales, Australia.

backgroundDigital proximity tracing apps were rolled out early in the COVID-19 pandemic in many countries to complement conventional contact tracing. Empirical evidence about their benefits for pandemic response remains scarce. We evaluated the effectiveness and usefulness of COVIDSafe, Australia's national smartphone-based proximity tracing app for COVID-19.

methodsIn this prospective study, done in New South Wales (NSW), Australia, we included all individuals in the state who were older than 12 years with confirmed, locally acquired SARS-CoV-2 infection between May 4 and Nov 4, 2020. We used data from the NSW Notifiable Conditions Information Management System, the national COVIDSafe database, and information from case interviews, including information on app usage, the number of app-suggested contacts, and the number of app-suggested contacts determined by public health staff to be actual close contacts. We calculated the positive predictive value and sensitivity of COVIDSafe, its additional contact yield, and the number of averted public exposure events. Semi-structured interviews with public health staff were done to assess the app's perceived usefulness.

findingsThere were 619 confirmed COVID-19 cases with more than 25 300 close contacts identified by conventional contact tracing during the study period. COVIDSafe was used by 137 (22%) cases and detected 205 contacts, 79 (39%) of whom met the close contact definition. Its positive predictive value was therefore 39%. 35 (15%) of the 236 close contacts who could have been expected to have been using the app during the study period were identified by the app, making its estimated sensitivity 15%. 79 (0·3%) of the estimated 25 300 contacts in NSW were app-suggested and met the close contact definition. The app detected 17 (<0·1%) additional close contacts who were not identified by conventional contact tracing. COVIDSafe generated a substantial additional perceived workload for public health staff and was not considered useful.

interpretationThe low uptake of the app among cases probably led to a reduced sensitivity estimate in our study, given that only contacts who were using the app could be detected. COVIDSafe was not sufficiently effective to make a meaningful contribution to the COVID-19 response in Australia's most populous state over a 6 month period. We provide an empirical evaluation of this digital contact tracing app that questions the potential benefits of digital contact tracing apps to the public health response to COVID-19. Effectiveness evaluations should be integrated into future implementations of proximity contact tracing systems to justify their investment.

fundingNew South Wales Ministry of Health (Australia); National Health and Medical Research Council (Australia).