Evaluability assessment of a targeted national anti-child poverty policy: option appraisal for evaluating the Scottish Child Payment.

backgroundThe Scottish Child Payment (SCP) is a recurring cash transfer introduced in 2021 for low-income families, unique within the United Kingdom and forming a core part of the Scottish Government's statutory child poverty reduction strategy. As a long-term, population-level intervention, the SCP presents an important opportunity to generate evidence on the impact of cash transfers. However, multiple policy changes have made evaluation design challenging. We conducted an evaluability assessment (EA) to identify feasible, policy-relevant evaluation options.

methodsOur five-stage EA approach included: (1) EA inception, (2) developing a theory of change, (3) prioritizing outcomes and identifying available data, (4) developing evaluation designs and (5) appraisal and reporting. Two stakeholder workshops, informed by evidence reviews, were held. The first refined the Scottish Government's theory of change. The second focussed on identifying priority outcomes, available data sources and suitable evaluation designs. Stakeholders reviewed a final report summarizing these options.

resultsGiven extensive existing and planned qualitative research, stakeholders agreed that a quantitative evaluation should be given precedence. They agreed that child development and child and maternal health and wellbeing outcomes should be prioritized and measured using administrative data. Due to data access constraints, SCP exposure would be estimated using eligibility indicators rather than direct receipt. Evaluating effects on health inequalities for Scottish Government-identified priority family groups compared with the rest of the population was considered important, though linkage of household-level benefits data remains limited. We identified four feasible natural experimental designs: before-and-after analysis, interrupted time series (ITS), controlled ITS and regression discontinuity design (RDD). Each design may be suited to different questions, populations and outcome types, with multiple designs likely to be helpful.

conclusionsThis EA demonstrates that a robust quantitative evaluation of the SCP is both valuable and feasible, but must be tailored to specific outcomes, implementation phases and evolving policy contexts. Linked administrative data offer strong potential, though advocating for more precise measures of SCP receipt and household socioeconomic context to explore inequalities should continue. Estimating the SCP's effect on child and maternal health should be a priority to inform the design of future poverty reduction policies.