Evaluating self-reported vaccination hesitancy in mobile phone surveys in low- and middle-income countries: learned lessons from Ethiopia, Indonesia, Kenya, and Malawi.

Background: The large amount of data on COVID-19 vaccination hesitancy presents a unique opportunity to better understand COVID-19 vaccination uptake. However, the utility of this data is unclear, particularly how representative the surveys are of general populations, how easy the data is to use, and how valid the outcome (intent to be vaccinated) is. We explored this in the World Bank's high frequency phone surveys (HFPS). Methods: The HFPS were conducted longitudinally in over 50 countries between 2020-21. A subset of the HFPS contained questions on vaccination hesitancy. We compared the demographic results from four surveys against the most recent census to determine the representativeness of the sample and vaccination intent/actual vaccination against government-reported vaccination rates. Results: While the surveys were generally representative of population sizes and the rural/urban split, they tended to over-sample men and older people and omitted several key indicators. We also found that self-reported vaccination rates were higher than actual vaccination rates. Conclusions: It is important to consider challenges in the HFPS data and other datasets which measure vaccination acceptance by phone surveys. It is also important to consider the ease of data use. However, even when these challenges arise, there are still opportunities for meaningful use of the data.