Measuring Impact

Research

Measuring ImpactAchieving quality is Bastow’s overarching aim. The quality of our communication with participants and partners; the quality of our course materials and the quality of the learning experiences we provide.  How do we know we’ve achieved quality? What is the indicator for success? The answer is ‘impact’. Impact is the effect the learning experience has on the knowledge, skills and understandings of educational professionals, their classroom, team or school. When seeking to have an impact we are essentially trying to change behaviour, to improve the practice of educational professionals.

So how do we measure impact?

Impact is measured by collecting data from multiple sources. This is called triangulation and results in stronger, more valid conclusions about the degree of impact. Bastow collects data about courses through a combination of questionnaires, interviews and direct observation. Direct observation (a third party) is important because it provides a more objective and arguably more accurate assessment of changes in a person’s behaviour than the individual who undertook the professional development.

The major challenge with evaluating the impact of a course is establishing that it was indeed the course that led to the change in practice and nothing else. This is called attribution. Determining attribution is only possible through a strictly controlled experimental study, something which is rarely done in education. As a result most evaluations, including the Bastow evaluations seek to establish the contribution the course makes to changed practice. Contribution acknowledges that other factors influence changed practice including other professional development, different cohorts of participants, new teachers etc. Bastow works with a specialist evaluation company, Sinergistiq to assess impact.

Continuous Improvement

But impact is not the only reason we evaluate our courses. We also evaluate to continuously improve courses. Commonly known as formative evaluation, we draw on a range of sources of data including interim and final course questionnaires, real-time feedback, our facilitators and our own observations to continuously adjust and improve the learning experience. The main aim is to ensure that the learning experience is engaging, sets high expectations and is relevant to the needs of educational professionals.

Another important reason for evaluating our courses is to provide evidence for accountability in using public resources. Bastow, like other publicly funded organisations, has a responsibility to ensure funding is used in the most effective and efficient ways possible. Rigorous evaluation provides this evidence.