Objective
USAID/South Africa contracted Khulisa to conduct an external, mid-term performance evaluation of
the Sexual HIV Prevention Program (SHIPP) project. SHIPP was a 5-year technical assistance program
supporting multiple levels of the South African Government (SAG) as well as communities in selected
high-prevalence districts in South Africa to strengthen “combination HIV prevention” for reducing
overall HIV incidence.
The evaluation, covering SHIPP’s implementation from September 2010 to March 2014, aimed to: (i)
assess the quality of the program implementation, particularly in relation to support to SAG
departments, (ii) document lessons learned, (iii) explore challenges and accomplishments, and (iv)
provide strategic guidance for the program going forward.
Approach
This project illustrates Khulisa’s experience in conducting project performance evaluations in Africa.
Seven overarching evaluation questions guided Khulisa’s approach:
- To what extent has SHIPP achieved its stated objectives/results so far? What are the reasons
for any shortfalls? - To what extent have combination HIV prevention approaches been incorporated into policy,
planning, and strategy in partner SAG departments? - What are the strengths and weaknesses of the program so far?
- What key aspects of the SHIPP model should be continued in future HIV/AIDS programming?
What should be discontinued? What should be scaled up? - What systems have been established due to the technical assistance provided by SHIPP?
- What are the examples of working partnerships established with SAG structures?
- What aspects of small grant activities are successful to inform future prevention models?
A non-experimental design was used which took into consideration SHIPP’s direct and indirect contributions to observed changes. Khulisa selected elements of several evaluation theories for their utility in answering the evaluation questions:
- Collective Impact. Successfully addressing complex social problems (such as HIV prevention) across multiple partners is achieved through attention to 5 core domains and 7 additional domains (Table 1) that facilitate the process for collective change.
- Most Significant Change. This approach provides an indication of a program’s perceived impact and is useful for explaining how change takes place and under what circumstances.
Impact:
The evaluation's findings and conclusions included recommendations for PEPFAR/USAID for future HIV prevention programs. The following recommendations were made:
1. Strengthen Leadership and Governance:
- Continue to support leadership capacity at multiple levels (national, provincial, district) within South Africa’s government to enable strategic policies and programs for HIV prevention.
- Foster collaboration among diverse stakeholders, including government and civil society, to ensure alignment and shared ownership of HIV prevention efforts.
2. Enhance Policy Development and Operational Plans:
- Ensure the creation and alignment of HIV prevention policies with the National Strategic Plan (NSP) for HIV, STIs, and TB.
- Develop and implement operational plans at district levels to improve the strategic planning capacity of local governments.
3. Improve Service Delivery:
- Strengthen HIV prevention service delivery systems through technical assistance and secondment of staff.
- Expand support for community-level interventions, such as small grants to local organizations, to improve the quality and reach of services.
4. Promote Combination HIV Prevention:
- Focus on integrating biomedical, behavioral, and structural interventions into national and local policies.
- Prioritize operationalizing combination prevention approaches for greater effectiveness.
5. Address Challenges in Staff and Implementation:
- Mitigate the impact of high staff turnover by implementing strategies to ensure continuity in project teams.
- Consolidate efforts in high-prevalence areas to avoid spreading resources too thin and improve the program’s effectiveness.
6. Strengthen Monitoring and Evaluation (M&E):
- Develop and implement effective performance monitoring plans that include relevant PEPFAR indicators for technical assistance.
- Address critical capacity gaps and data backlogs in M&E systems.
7. Operationalize Policies:
- Focus on the adoption and implementation of policies supported by SHIPP to ensure their immediate and sustained impact.
Broader Implications:
- Scale-up future HIV prevention programs by operationalizing combination prevention, improving monitoring systems, and consolidating efforts in fewer, high-impact areas.
- Leverage strategic technical assistance and leadership development to address complex health challenges like HIV.