The best programmes work collaboratively with the evaluator to develop the M&E plans, activities and Theory of Change together, giving the Programme Staff immediate feedback on their plans and monitoring system. This collaboration allows the evaluator to work with the programme staff to come to mutual agreement about the Theory of Change, the log frame, the structure and quality of the monitoring system and key outcome indicators. This iterative process creates a better chance of demonstrable success (assuming there is an enabling environment for the programme).
Khulisa’s name has been synonymous with leading monitoring and evaluation (M&E) practice, throughout Africa since 1993. We provide support to governments, the donor community, NGOs and the private sector to improve institutional monitoring capacity, which in turn improves the potential for evaluation. Capacity building is not new to Khulisa. As leaders in the field, we aim to improve programme evaluations, and empower M&E professionals through direct and indirect capacity building.
We promote stakeholder learning in an attempt to clarify, demystify and institutionalise M&E practices and find that our audiences are increasingly becoming more mature and articulate in their understanding and requirements for M&E. Well over half (57% or 45 of 79) of our assignments have M&E capacity building as an explicit project goal.
Signing on the Same Page
M&E does not exist in a vacuum; rather it comprises various building blocks to form the big picture and requires connected participation – at all levels – to be effective. Developing an M&E system, regardless of the methodology used, needs all parties to establish and agree on a shared definition.
NGO or government staff may not have shared understanding of critical monitoring terms, or explicitly understand their own programme’s Theory of Change. Working through a collaborative process with these staff allows Khulisa to surface hidden programme assumptions about programme effect and outcomes. The resulting M&E systems therefore assist the programme to collect and analyse data to:
- Reflect and improve programme implementation
- Elicit learning
- Ensure accountability
- Develop knowledge
Any M&E system is a learning strategy; it has to be systematic. It needs to promote quality, accountability and professionalism, and should come from a place of respect. An effective system will adopt the needs and interests of all stakeholders: beneficiaries, partners, funders, as well as the public. These constituencies have different agendas. It therefore requires sensitivity and listening before framework design commences.
Buy-in, inclusivity and ownership is paramount. Khulisa recently developed an M&E framework for an urban permaculture project and gained input from all staff levels. People responded, because they felt engaged.
The gardeners who attended the workshop were proof of the value in shared participation. Dressed up in their suits, they participated with the most enthusiasm of all the groups. Their inputs were refreshing and useful for the development of the framework, while they left with a sense that they were part of the planning, which they never have before. They understood the rationale for M&E, and made it their own.
In our capacity building efforts, we learned that people with little to no M&E experience were able to participate with equally valuable inputs as that of experienced evaluators and M&E professionals. In fact, directors and programme staff often over-think to a point where it is difficult to develop a strategy.
Using Light Touch Monitoring Techniques
Too often programme staff think they need to develop new questionnaires and surveys every time they want a piece of data. This often results in fatigue by respondents, fieldwork staff and managers. We encourage our partners to look for light touch data points.
For example, Khulisa is now using social media analytics as a proxy indicator, such as the popularity of a Facebook Post or Tweet to demonstrate message reach and linkages. Other light touch techniques include mapping using the GPS signals from cell phones; holding competitions among beneficiaries; and creating weighted indices of already collected data.
Communicating Theory of Change, M&E Findings and Reports
The development sector is undergoing a communication revolution and communicating evaluation results in visual, easily accessible forms is increasingly important.
Khulisa is working with partners to create infographics, interactive maps, videos and even TED Talks to communicate evaluation findings in accessible, easily digestible formats. Such communication techniques increases ownership of evaluation findings and other outcomes.
When findings are presented to stakeholders during Khulisa’s “Recommendations Workshops” they are more digestible. Stakeholders then develop their own recommendations rather than impose external recommendations which builds ownership and improves the likelihood of implementing those recommendations.