Logic models for the evaluation of complex interventions in public health: lessons learnt from a staged development process.

backgroundLogic models are valuable tools when evaluating complex interventions in public health. They can improve the understanding of how an intervention works and interacts with its system, and facilitate clear communication regarding the intervention with stakeholders. There are different approaches to developing logic models. However, experiences with logic model development in practice, and the advantages or challenges associated with different approaches are rarely reported. This study describes and reflects on the staged development process of a logic model for the municipal public health intervention Präventionskette Freiham in Munich, Germany. Specifically, we aim to identify advantages and challenges associated with this process, and deduce lessons learnt for staged logic model development and logic model development in general.

methodsFor the staged logic model development process, a first draft of the logic model was developed at the start of the evaluation. We then defined a priori milestones at which the model was to be revised. It was revisited and updated: (i) after the process evaluation, (ii) after a workshop with stakeholders, and (iii) at the end of preparations for long-term outcome evaluation. We discussed the advantages and challenges associated with the staged development process in a workshop within the research team, and obtained feedback on the usefulness of the logic model during a workshop with municipal stakeholders.

resultsThe logic model changed in multiple aspects during the different stages of development, mostly due to evidence obtained during the evaluation and as a result of feedback from discussions within the research team and with stakeholders. In the workshop with the research team, we found the staged process to be useful for facilitating reflection, for increasing plannability of logic model development, for integrating multiple perspectives and for increasing the validity of the logic model. Challenges were the need for rigorous documentation and flexibility in the process, as well as the management of resources. In the stakeholder workshop on the usefulness of the logic model, participants stated that the graphical visualization made it easier to handle complexity, and wished for the logic model to be re-used for future interventions.

conclusionsDeveloping a logic model in a staged process over the course of the evaluation of an intervention is a useful approach for stimulating reflection and obtaining a more realistic depiction of the intervention.