Using qualitative information in evaluation

Using Qualitative Information in Evaluation

“Not everything that counts can be counted” Albert Einstein once pointed out.  Let's take the example seen in the love and care that an African grandmother (often called a ‘gogo’ in South Africa) shows to a young orphan and try to imagine what the impact of this love will be on the life of this child. (For more information see the goGogetter programme, started by local NPO loveLife and adopted by various other community organisations).

If we say that a large aspect of M&E is measuring whether something has happened, and if measuring implies counting and comparing quantities at different points in time, what role does that which we cannot count play in evaluation? 

In the example above, loveLife tried to establish the level of vulnerability of the orphan before a gogo would start interacting with him or her. This was done using a few practical indicators. Is the child attending school? Does the gogo suspect abuse? Is his family accessing a child support grant for the child's welfare?  loveLife then monitored the practical things that the gogo did to assist the child with the factors mentioned. For example, they assessed whether she would speak to the school headmaster and motivate for the child to return to school and whether or not she explained the requirements needed to claim the support grant for the family.  After a few years loveLife would check if the original vulnerability score for the child had changed, i.e. was the child less or more vulnerable when compared to the period before the gogo was involved? For more information read the learning brief on loveLife’s findings here.

The above worked fairly well to establish whether the programme was doing what it was meant to be doing effectively. However, the most important impact of the programme (that of the love experienced by the child) cannot be counted or illustrated in quantitative terms.  We know from research on early childhood development that early love creates a legacy of resilience in children which benefits them for a lifetime.  The field of epigenetics illustrates that early love and caring actually changes the way that our DNA expresses itself. This has been dramatically illustrated through studying the effects of early deprivation on brain development.

By studying the relationship between the gogo and the orphan in a qualitative way, we can begin to understand what that relationship looks like and what it means to the gogo and the orphan.  Over time we can gather a great variety of information on the child, ranging from school and clinic report cards and interviews with teachers and family members. We might even observe and interview the child at certain points in time.  As we study this information in depth, the meaning of the early care and love might start to reveal itself in subtle ways. When the child is older he or she might be able to reflect and share with us how the experience of that love and care played an influence and what he or she thinks it may have meant to her life trajectory.

It is not viable do such an in-depth study of the lives of many participants. However, when qualitative research is done well, we are able to draw valid conclusions when we are evaluating programmes.  The type of questions being answered are different and allow for a deeper understanding of the situation with greater insight into  why and how something is the way it is.