How should we evaluate the impact and implementation of palliative care education programs? The Pallium Canada experience.

Considerable effort and resources are invested in developing, implementing, and sustaining educational programs in palliative care, making evaluation an important component of these programs. Several frameworks, models and approaches are available to palliative care educators to inform their evaluation, and accompanying research activities. This paper describes the process used to develop a practical evaluation framework for Pallium Canada's educational programs. The objective was not to produce another comprehensive review of evaluation frameworks, as several reviews already existed, but rather to identify a practical and fit-for-purpose evaluation approach that could be implemented rapidly across Pallium Canada's expanding educational initiatives. A multiphase, multipronged approach was used, which included an environmental scan and input from a panel of national and international education and evaluation experts. Several trends in education evaluation were identified, including: a need to a) evaluate impact beyond learner feedback and acquisition of competencies, b) evaluate implementation alongside impact and outcomes, and c) approach education evaluation from a complexity lens. Twenty-one frameworks (including models and approaches) were shortlisted (and are summarized in the paper). They are organized in four categories based on their focus: a) Impact; b) Implementation; c) Economic evaluation; and d) Combinations. Several additional concepts, including pragmatism, quality improvement, and theories of change serve as useful lenses with which to approach evaluation. The final framework incorporates two main frameworks, namely the New World Kirkpatrick Model and the Consolidated Framework for Implementation Research. It also draws elements from other frameworks and approaches.