What should you evaluate?

What Should You Evaluate? 

There are various definitions for evaluation, but the core meaning is to assess something in a systematic way in order to understand its value. This implies learning about its strengths and weaknesses which is information that can be used in various ways to adjust or make decisions about the programme.  At a practical level evaluation comes down to asking simple but critical questions to enable us to learn and become accountable.

Accountability is often associated with accountability to funders. While there certainly is some accountability to funders, of most importance is the ethical responsibility organisations have towards their beneficiaries, to ensure that programmes really are meaningful. In most cases ‘beneficiaries’ are not just beneficiaries of programmes, but participants and stakeholders who have made an investment in the programme. This might be through labour, active support and participation or simply that they invest hope in improved living situations or life chances. 

There are many questions that can be asked and aspects of a programme that can be evaluated to gather useful information and insight. However, in terms of ethical accountability, it is important that organisations ask and seek answers to the following (evaluative) questions:

  • Is the programme effective?

  • How effective is it?

  • Why is it working or not working? (or why did it work or not work?)

  • Were there unintended side effects?

  • Who benefited most? 

  • Who was harmed?

In order to answer these questions an evaluation will need to look at:

a)      The impact of the project

This refers to the longer term effects of the programme beyond immediate results.  This is likely to be a comparison of a group of people who participated in, or were beneficiaries of the programme, with a group of people who were not involved.  In order to understand why the programme is not having an impact, the evaluation will also need to assess the achievement of each of the outcomes in the outcomes chain.

b)      The programme implementation processes (monitoring) 

This is an assessment of whether the programme is being implemented as it was designed to be implemented. This is an ongoing monitoring system for the programme and helps determine whether  or not the milestones and deliverables are on schedule.

c)      The programme theory

This assesses whether the programme theory is sound and the proposed actions or approach are able to lead to the stated outcomes. This will also contribute towards an understanding of why a programme is, or is not, effective.