Indicators: Basic Concepts

Indicators: Basic Concepts 

What are indicators?

We cannot directly observe social phenomena and yet know social phenomena exists and is present or absent under certain circumstances.  How do we know this?  There are indicators. Let's look at the concept of social cohesion as an example.  We know that social cohesion is present or absent by looking at  what people report about trusting others in their community;  how satisfied they say they are with life (supported by the suicide rate); the presence or absence of pro-social behaviour (such as volunteering, helping strangers or donating money); as well as voting rates, as high voting rates could indicate that people care and want to participate in the management of their community/country.

Attributes/Characteristics of Indicators

They are imperfect

Perhaps you can point out more indicators of social cohesion or you would like to challenge the ones that have been listed above.  This is an important aspect of indicators - they are approximations, imperfect and vary in validity and reliability[1].

Sometimes a single indicator is not sufficient to say that a certain change has occurred or that a phenomenon is present or absent. For example, a high voting rate on its own does not necessarily imply high social cohesion; it might be the opposite if people have been coerced to vote. In such cases you will need to include additional indicators and develop a set which together illustrate the existence (or absence) of the phenomenon.    

They can be shared

Indicators are sometimes obvious. For example, an indicator for reduced school drop-out rates would be the school graduation rate.  Other times indicators can be complex and require a lot of research to identify/develop the indicator/indicator sets.  It is in our interest to develop standard indicators of social phenomena for people to measure.  This allows us to compare information across different programmes, geographical areas or implementation circumstances.  You will see that in many fields, such as health or economics, standard indicators have been developed and it is only a question of selecting an appropriate one.  If you are doing something that is unique in nature, you will need to develop your own indicators. See the Tried and Tested section of this step for more information. 

You can look for indicators shared by organisations in the indicator library available under the tool section of this step.  If appropriate, you can use the indicator as is or adjust it and use it as a guide for developing your own.  Please be sure to contribute your indicators to this communal library.   

The purpose of indicators is to measure something

We look to indicators to answer the question ‘is it there’ or ‘did it happen’? Thus, the point of using indicators in evaluation is to help us observe and measure change. Did something happen that did not happen before; to what extent did it happen (was the change big or small); how often is it happening (is the change sustained or not) etc.? To answer these questions implies the counting and comparison of quantities at different points in time. Therefore indicators are generally expressed in terms of numbers or percentages (number of…; percent of…; ratio of…; incidence of…; proportion of…). Researchers would say it is quantitative.

Sometimes people refer to qualitative indicators when they are measuring phenomena which they would define as 'qualitative' (such as being happy) and to quantitative indicators when they are referring to something they can obviously count (like the number of brochures that they have distributed). The reason why indicators are generally expressed in a quantitative terms is not related to the nature of the phenomena that you are investigating however, it is because the type of questions that we are trying to answer through indicators requires counting (it happened yes (1) or no (0); it happened to a large extend (90%) or it barely happened (3%). That does not mean that qualitative information is not useful to evaluation, it answers different questions though and therefore it is not expressed as indicators (read here about the usefulness of qualitative investigation in evaluation). It does mean that when you are selecting indicators you need to be sure that there are ways of measuring (counting) the occurrence of the indicator – for example, levels of happiness are generally established through life satisfaction surveys.

Indicators are complemented by:

 

- targets
- a baseline
- comparison     group

 

Targets (to establish the level of change needed to qualify success)

How much change will be enough to achieve the impact that you want your programme to achieve? If you offer a skills training course that capacitates people to find employment, is it sufficient if only two out of ten find employment? Or would you say your programme has achieved this outcome when at least seven or eight out of ten people find employment? Because change and social phenomena can occur in ‘quantities’ (ranging between a lot and a little), it is important that you qualify at which level of change you would say that your programme has successfully achieved (the outcome(s)).  Because you are using an indicator to establish (measure) whether the change has happened or if certain social phenomena are occurring, your targets need to relate to your indicators.  

Baseline (to establish if change has happened and to inform targets)

If you want to establish whether something has changed, you need to compare it to how it was before your intervention.  To do so, you need to know the initial status before your intervention – we call this a baseline.  Basically, a baseline requires you to do all the measurements, for selected indicators of outcomes that are key to showing that the intervention is achieving what it set out to do, before you start implementing the programme. You can keep those numbers on record for future comparison when it comes to evaluation and you can also use it to inform your targets.

A comparison group  (to establish if your programme is responsible for the change)

It is fine to establish whether change has happened using indicators, targets and baselines, but how do you know that this change happened as a result of your intervention? In some cases, it is obvious and there might be no need to investigate further.  This might be seen in a programme that establishes a children’s hospital in a needy area and as a result more children are receiving medical care when they need it.  However, interventions with less concrete outcomes makes it more difficult to report that the programme is responsible for or has contributed to the change. For example, a programme designed to improve a love of reading might measure the number of books that children say they read in a month (an imperfect indicator, but we will use it for the sake of an easy example). If the programme notices an increase in the number of books read compared to the baseline, it could be as a result of their programme. However, if the SABC had started broadcasting a programme promoting reading, or the Department of Education improved school libraries, we cannot really be sure that the ultimate change was solely due to the example programme given. The best way to establish attribution would be to look for the occurrence of your indicators in two groups: the target group that you have been working with and a group that you know for sure has had no exposure to the programme. Researchers would call the second group a ‘control group’. This process of comparison is called quasi-experimental or true experimental research.