Objective
USAID/South Africa contracted Khulisa to conduct an end-of-project program evaluation of the
USAID/South Africa TB project for developing a follow-on project to support South Africa’s National
TB Programme (NTP).
The evaluation is to answer five evaluation questions:
- To what extent did the project achieve its intended results as stated in the contract’s
objectives? What were the reasons for any shortfalls? - To what extent is the design of this project valid? How successful have been the
programmatic and management approaches, structures and systems in carrying out the
project’s activities? - Did the project strengthen the capacity of NTP sufficiently to ensure its sustainability?
- What role has the Small Grants program played in improving TB, TB/HIV and MDR TB
programs? - What strategies were used to reach vulnerable populations? E.g. Mines and work place
programs. - How has the project integrated technology e.g. mobile health work in KZN, Active TB case
finding?
In answering these questions, the evaluation sought to: (i) assess the quality of the project’s design,
technical inputs, and implementation to improve TB outcomes; (ii) determine which approaches and
activities were employed, which were successful and why; (iii) assess the effectiveness of the project
in strengthening TB prevention and control in the Republic of South Africa (RSA);(iv) identify actual
outcomes achieved; and (v) summarize key results and effectiveness of the project in achieving its
intended objectives.
Approach
To answer the evaluation objectives, the Khulisa team used a non-experimental design and a mixed
methods approach to collect both primary and secondary data and to ensure that the evaluation
questions were answered comprehensively. The methods employed included document and data
review; analysis of TB performance in project supported areas vs non-supported areas; key
informant interviews; and an online survey.
Pre-fieldwork
- Briefing: Preliminary discussions (prior to arrival in country) with the USAID/SA management team to review SOW, agreed on key evaluation questions, evaluation design and data collection methods, finalized schedule and logistics.
- Desk Review: The evaluation team conducted an in-depth review of background documents and relevant materials such as Project quarterly, semi-annual and annual reports, Work plans and any management reviews, M&E plan, etc., selected project research and technical reports, publications, and tools
Fieldwork
- Key Informant Interviews: The evaluation team conducted qualitative, in-depth, and structured interviews with key stakeholders, partners, and beneficiaries such as URC leadership and staff; USAID/SA management; national, provincial and district department of health representatives; U.S.G and other international partners; local implementing partners and service providers, as well as project beneficiaries.
- Field Visits: The evaluation team conducted field visits at each of the provinces and districts where the project has been operational.
Impact:
The evaluation served as a critical feedback mechanism to refine program strategies, enhance
sustainability, and ensure alignment with national and global TB control goals. The evaluation
identifies several impacts resulting from conducting this evaluation:
- Informed Planning for Future Projects: The evaluation provided insights to inform the
design of a follow-on project to support South Africa’s National TB Program (NTP). This
includes recommendations for better resource allocation, strengthening capacity building,
and improving community-based interventions. - Improved Understanding of Project Outcomes: The evaluation assessed the project’s
effectiveness in achieving its goals, such as improved TB treatment success rates and
integration of HIV and TB services. It highlighted areas of success and gaps, helping
stakeholders understand the project’s contributions and limitations. - Enhanced Stakeholder Collaboration: The evaluation process involved extensive
engagement with stakeholders, including government officials, healthcare workers, and
community organizations. This fostered collaboration and encouraged collective efforts to
address TB challenges. - Identification of Best Practices: The evaluation documented successful strategies and
approaches, such as the use of small grants for community engagement and innovative tools
for TB management. These practices can be scaled or adapted for similar health programs. - Capacity Building: The evaluation process itself contributed to capacity building by training
local stakeholders in monitoring, evaluation, and data analysis. - Policy and Strategic Recommendations: The evaluation provided actionable
recommendations, such as including the National Health Laboratory System (NHLS) as a
formal partner and pursuing technological innovations for TB management.