Projects
Featured Projects
South Africa: Baseline assessment and end-term evaluation for FNB’s Primary Education Programme
Africa and India: Conducting Scoping and Field Research Studies for GALVmed
Evaluating the smallholder farmer sector for the South African Presidency
All Projects
PEPFAR/South Africa Landscape Analysis: Current and Potential Role of South African Faith-Based and Traditional Structures to Support the HIV Response
Description:
Objective:
With the world’s largest antiretroviral therapy (ART) program, South Africa has seen significant improvements in HIV morbidity and mortality, as well as in quality of life over the last two decades. However, South Africa realized it could only reach its 90-90-90 targets by 2020 through effectively engaging respected and trusted community-based entities (such as faith-based organizations [FBOs], traditional leaders, traditional healers, and unions) who are uniquely positioned to support their communities with prevention, treatment, and care services.
USAID/South Africa, through FHI360, commissioned Khulisa to conduct a comprehensive landscape analysis of FBOs, traditional structures (TSs), and unions that were currently engaged in, or who could potentially be engaged in, the HIV response in South Africa, especially those who operate in high-priority districts supported by USAID and PEPFAR.
The objectives of the landscape analysis were to:
- Identify FBOs/TSs/unions offering HIV services or programs in 10 high-priority districts of South Africa.
- Determine the extent to which FBOs/TSs/unions could:
- Mobilize demand for ART;
- Actively link people living with HIV (PLHIV) to ART;
- Support ART adherence and retention; and
- Increase demand for HIV services.
- Identify gaps and needs of FBOs/TSs/unions in scaling-up ART among PLHIV and among their constituents or in their communities.
Approach:
Khulisa initially identified approximately 60 FBOs/TSs/unions operating in South Africa and shortlisted these by their likely ability to succeed if engaged by USAID to implement an HIV program. The shortlisting approach was designed against several criteria based on discussions with USAID and FHI360:
- The organization was currently implementing an HIV program in South Africa, and the current HIV program:
- Worked with adolescent girls and men aged 19-45 in some capacity;
- Worked in one of the 10 priority PEPFAR districts;
- Had technical and management capacity to expand its activities to referring PLHIV to health facilities for ART (if not already doing so) and supporting PLHIV on ART to stay on ART;
- Had established indicators for reporting on program outputs, a track record of regular reporting, and a functional data management/M&E system;
- Had a track record of managing donor funds and had adequate financial systems in place to account for external financing; and
- Appeared interested in expanding into PLHIV treatment support activities.
Khulisa undertook three separate data collection activities to identify the final shortlist of organizations:
- Phase 1: Brief telephone interviews (less than 20 minutes) focused on identifying the existence of any HIV-related program, the main activities, the main target groups for those activities, the districts where the HIV program was operating, and contact details for key HIV program respondents. Interviewers also identified names of other FBOs/TSs/union organizations that were implementing HIV programs. Based on the results of this phase, a preliminary shortlist of organizations was created for phase 2.
- Phase 2: Key informant interviews with head office staff or the FBO, TS, or union in the preliminary shortlist. The interviews, which were no more than two hours and conducted face-to-face or telephonically, focused on obtaining more detailed information about the HIV program’s: 1) activities, target groups, partnership structures, approaches to engaging communities in HIV prevention and PLHIV support services, and operating locations; 2) program and organizational management structures, including staffing profiles, use of volunteers, and how volunteers are supported; 3) funding profile and financial management structures; and 4) names of other FBOs/TSs/union organizations that were implementing HIV programs. Based on the results of this phase, a further shortlist was created for Phase 3.
- Phase 3: In-depth, full-day assessments at head offices and/or implementation sites, focusing on assessing five key areas of organizational capacity (measuring 20 different elements of program design and management) using an adaptation of the McKinsey NGO organizational assessment tool. Organizations were scored against 20 capacity elements using a rubric describing organizational capacity.
Impact:
Landscape analysis findings provided a final shortlist of potential organizations that PEPFAR could engage for supporting South Africa’s ART program. The landscape analysis also offered additional recommendations for enhancing community-level interventions to assist South Africa in reaching the 90-90-90 goals:
- Leverage the influence of traditional and religious leaders to address HIV-related stigma and improve the uptake of HIV services;
- Increase HIV and ART literacy among PLHIV, community cadres, and stakeholders to improve early ART enrollment;
- Strengthen and expand community-level (especially FBO) adherence support groups and ARV dispensing points;
- Strengthen relationships between community entities and the Department of Health/Government;
- Harmonize stakeholder implementation and support collaboration;
- Better target male and other key population groups; and
- Create guidelines for community-level activities around the 2nd and 3rd 90s.
Key Staff:
Mary Pat Selvaggio, Team Leader
Puveshni Crozier, HIV Expert and Evaluator
Kris Eale, Evaluator
David Ndou, Jr. Evaluator
African Strategies for Health (ASH)
Description:
Objective:
The African Strategies for Health (ASH) Project was a five-year contract funded by the United States Agency for International Development/Africa Bureau (USAID/AFR). It was implemented by Management Sciences for Health (MSH) in partnership with three Africa-based partners: African Population and Health Research Center (APHRC), Khulisa Management Services, and Institut pour la Santé et le Développement (ISED) of Dakar University, Senegal.
ASH’s mandate was to assist USAID/AFR to work with African Institutions and other development partners to guide USAID investments for furthering the health of Africans. The ASH project conducted reviews, assessments, and dialogues with partners to improve the understanding of constraints impeding the reaching of the Millennium Development Goals (MDGs) and the goals of the Global Health Initiative (GHI).
Approach:
Khulisa provided Monitoring and Evaluation (M&E) support to the ASH project and to project beneficiaries Khulisa’s key M&E responsibilities were to develop the M&E Plan, provide M&E and research expertise (including employing an M&E Advisor) across all portfolios of the project, and carry out supplemental M&E related activities for the project.
The M&E advisor sitting on the core ASH management team was a Khulisa employee. In addition, Khulisa provided short-term M&E consultants.
Impact:
During Year 1, Khulisa’s focus was on assisting with the Project Management Plan (PMP), M&E plan, and developing M&E building blocks for ASH activities.
Year 2 saw the beginning of the formative stages of Khulisa’s main contributions to ASH. Khulisa played a key role in conceptualizing and conducting a landscape analysis of health-focused regional organizations. In Year 3, ASH, through Khulisa, worked closely with the African Evaluation Association (AfrEA) supporting the 7th AfrEA International Conference (held March 2014) by organizing, coordinating, managing, and sponsoring the conference’s health strand. Khulisa (i) conceptualized the health strand sessions, (ii) advertised for health strand abstracts, (iii) gathered and selected abstracts (of 325 abstracts submitted for the health strand, 48 abstracts were accepted), (iv) provided technical guidance to evaluators/presenters in preparing their poster and oral presentations, (v) identified and recruited the keynote speaker and key presenters, (vi) assembled and chaired panel discussions to address issues related to health evaluation, and (vii) co-designed and facilitated a knowledge sharing and skills building workshop on “The USAID Evaluation Policy: Quality, Standards, Lessons Learned and Experiences”.
The AfrEA-ASH/Khulisa collaboration continued in Year 4 with the development of a Special Edition of the African Evaluation journal (AEJ) dedicated to “Health-Focused Evaluations in Africa” (http://www.aejonline.org/index.php/aej/issue/view/7). Khulisa worked closely with the AEJ Editor-in-Chief to develop a collection of articles about health evaluations conducted in different regions of Africa, authored by both emerging and seasoned evaluators from Egypt, Nigeria, Guinea, South Sudan, Uganda, Tanzania, and Madagascar. Khulisa also provided capacity building to the authors in writing M&E journal articles.
In Year 5, ASH/Khulisa initiated a new collaboration with the Pan African Parliament’s (PAP) Committee on Health, Labor, and Social Affairs as part of the Global Parliamentarians Forum for Evaluation (GPFE) “Parliamentarians for Evaluation” Campaign. Also in Year 5, ASH/Khulisa provided technical support for developing an assessment tool for the “SADC Regional Minimum Standards for HIV and Other Health Services along Transport Corridors” also known as the SADC RMSB. This activity built on earlier ASH support provided to SADC for creating minimum standards for HIV and other health services at the borders of SADC countries that cater to long distance truck drivers and sex workers who often lack access to essential health services from national programs.
Key Staff:
Lungi Okoko – M&E Advisor
Tabitha Kibuka – M&E Advisor
Mary Pat Selvaggio – M&E Strategic Support
Salome Omolo – Government Liaisons
Zach Akiy – Epidemiologist
Consultants
Diagnostic/Implementation Evaluation of Nutrition Interventions for Children from Conception to Age 5 for the South African Government in South Africa
Description:
Objective:
This evaluation assessed the implementation of 18 nutrition interventions delivered by four ministries of the South African Government -- Health, Social Development, Agriculture, and Rural Development – and sought to determine the enabling and inhibiting factors for implementation.
Approach:
Khulisa used a qualitative case study approach to: understand the Government’s underlying program theories around its nutrition interventions; evaluate the effectiveness of project implementation; identify barriers to effective delivery; and gauge fidelity to relevant policies. The evaluation examined critical system and implementation issues that inhibit or enable access to nutrition interventions for pregnant women and child under the age of five.
Data collection methods included a literature review; in-depth interviews; focus group discussions with key respondents at national, provincial, district, and site levels; and a rapid performance assessment administered at health facilities and community nutrition programmes.
Through both the literature review and fieldwork in four provinces of South Africa – KwaZulu-Natal, Western Cape, Free State, and Eastern Cape – Khulisa broadly examined the implementation of the 18 nutrition interventions. We undertook in-depth analyses for eight case studies: four provincial case studies and four case studies on the following priority interventions:
- Breastfeeding support;
- Targeted meal supplementation (delivered through health facilities); and
- Food access (food parcels, food banks, and food kitchens delivered by the Department of Social Development), and household food production and preservation (home gardening).
Impact:
The results of this evaluation had a direct policy impact. South Africa’s National Food and Nutrition Security Plan (2018 to 2023), a comprehensive and integrated national plan, specifically mentions the findings from Khulisa’s evaluation and attributes the development of the national plan to the evaluation’s findings.
Key Staff:
Mary Pat Selvaggio
Edna Berhane
Safety and Security Sector Education and Training Authority (SASSETA) – Systems Improvement and Culture Change Project
Description:
Objective:
The Safety and Security Sector Education and Training Authority (SASSETA) is one of 21 sector education and training authorities (SETAs) in the South African government responsible for skills development and implementation. SASSETA specifically focuses on developing and implementing skills within South Africa’s safety and security sector.
The South African Government Technical Advisory Centre (GTAC) contracted Khulisa to support SASSETA with mentoring, leadership/team coaching, systems improvement, and culture change services to support SASSETA’s turnaround strategy toward becoming a performance-driven organization.
Approach:
Khulisa’s support focused on improvements in three organizational areas:
- Human resource management (HRM);
- Supply chain management (SCM), including related internal audit and control processes and contract management of discretionary grand-funded projects; and
- Monitoring and evaluation (M&E) for performance management.
After assessing SASSETA’s operational systems and organizational culture, Khulisa designed and executed a plan for individual coaching of senior SASSETA leadership and group coaching of three SASSETA teams to improve management systems and processes in the three areas indicated above. The overall approach was to support HR and SCM staff to drive their own capacity-strengthening, in order to build commitment to improvements at individual and organizational levels.
Impact:
As a result of Khulisa’s support, SSASETA managers successfully:
- Updated and produced several policy documents and standard operating procedures;
- Developed new and amended tools for M&E, SCM, and performance management;
- Institutionalized selected management processes, such as using evidence for performance management and for quarterly reporting; and
- Addressed and reduced Auditor General findings.
Key Staff:
Solome Omolo
Puveshni Crozier
EGPAF/Mozambique Data Quality Audit of HIV/AIDS Performance-Based Financing Program in Maputo, Gaza, Nampula, and Cabo Delgado Provinces
Description:
Objective
The Elizabeth Glaser Paediatric AIDS Foundation (EGPAF) and its local affiliate, Fundação Ariel Glaser (Ariel), support prevention of mother-to-child transmission (PMTCT), HIV Care, and antiretroviral treatment (ART) services at numerous sites in Mozambique. Some of these sites are funded through performance-based financing (PBF) contracts.
EGPAF commissioned Khulisa to conduct a comprehensive external Data Quality Assessment (DQA) to confirm the quality of data reported on 21 PBF and 4 non-PBF indicators in four provinces of Mozambique. The DQA’s main objective was to identify data quality issues to resolve and strengthen the monitoring and evaluation (M&E) system. The DQA also assessed concordance (consistency) of data reports from PBF invoices and routine data collection systems.
Specifically, the DQA:
- Assessed the quality of reported data (using standard data quality criteria of validity, reliability, timeliness, precision, and integrity) in terms of its concordance with source registers submitted by EGPAF/Mozambique;
- Assessed concordance of data reports from PBF invoices and routine data collection systems;
- Estimated the margin of error of reported data vis-à-vis source data, by district and by province;
- Assessed the various data collection, processing, and management systems implemented by EGPAF/Mozambique;
- Assessed the flow of data from the initial point of data collection to higher levels of reporting;
- Identified areas of potential vulnerability that affect general credibility and usefulness of the data; and
- Recommended measures to address any identified weaknesses in the data submitted by EGPAF/Mozambique, as well as the M&E procedures and systems in place.
Approach:
Given that the quality of reported data is highly dependent on the underlying data management systems (DMS), Khulisa assessed and verified the DMS and data for EGPAF’s PMTCT and ART services in a sample of sites (service delivery points, data aggregation points, and head office operations). The DQA involved a desk review of project documentation and data and two procedures:
- An assessment of the DMS in five functional areas; and
- A physical verification of data reported in the defined reporting period against established data quality criteria.
Combined, these two procedures allowed Khulisa to a determine the adequacy of data management systems/processes, the quality of data produced within these systems/processes (against specified data quality criteria), and the adequacy of human and physical resources to produce quality data.
Both procedures include interviews with key staff involved with the DMS: M&E specialists/officers, nurses, data capturers, and people in charge of report development and submission.
Impact:
The DQA found that the EGPAF/Ariel data was valid and collected according to the definitions in the standard operating procedures. The main overarching recommendation highlighted the need for improvements in the following areas:
- Data Storage: Facilities should be provided with resources to protect registers and program data documents from being damaged by people or floods.
- Data Management: Establish a comprehensive data quality assurance plan that includes regular training for DMS staff at facilities to maintain accuracy and consistency in data handling.
- Data Management Personnel: Implement and regularly conduct mini workshops on standard operating procedures and guidelines dealing with data management.
- Technology: Introduce data capturing via a mobile device, in which information can be automatically uploaded to a cloud database, to mitigate the numerous shortcomings of the current data recording and collection system.
Key Staff:
Mary Pat Selvaggio, Project Director
Zach Akiy, DQA Expert
Salome Omolo, DQA Expert
Kris Eale, DQA Assistant
Research on the Nature and Scope of Trafficking In Persons (TIP) in South Africa
Description:
Objective:
There is growing recognition that labor exploitation of migrant workers has become a problem of global proportions. Human trafficking and other forms of extreme exploitation, including forced labour and forced marriage, now collectively under the umbrella term ‘modern slavery’, are reported to affect an estimated 40.3 million people globally, with 29.4 million considered to be in situations of forced labor.
The purpose of this research program was to make a step change in scale and impact by focusing on robust, empirically driven, and policy- and socially relevant studies, which were aimed at revealing the nature and magnitude of TIP in South Africa.
We envision that evidence-based studies will create a baseline and shed light on the complex characteristics of trafficking and will target the underlying drivers of exploitation.
This research program aims to ensure that data on human trafficking prevalence play a larger role in public policy debates, particularly given the hidden and subversive nature of human trafficking.
Approach:
The research team conducted a landscape review (desk review) of the TIP counter-trafficking policies, practices, research, and available data in 2020. Thereafter, we conducted a number of research studies using mixed-method approaches, including a questionnaire to key stakeholders to collect more recent data and a wider breadth of detail, and a scan of the existing databases held by various South African organizations.
Activities included:
- A thematic analysis of successfully prosecuted TIP cases (sexual, labor, and other types) under both the 2007 Sexual Offences Act 32/2007 and the Act 7/2013 legislation with concomitant interviews with available witnesses in these cases to mine additional insights into indicators of the crime, as well as the magnitude based on witness interactions with other victims and traffickers across multiple locations. View here.
- A report on TIP victim voices, both adults and children. Data was collected through Key Informant Interviews (KIIs) with adult victims to capture lived experiences before, during, and after trafficking. These interviews captured vulnerabilities and gaps in services and to identify aspects that need addressing, identify what is working or beneficial, critical aspects, and interventions that could be up scaled A further study included reviewing child victim case files to gather data on their trafficking experiences and response services. A presentation on trafficking of children is available here.
- A final research report on the nature and scope of trafficking in South Africa. Components of the report included a summary of the above reports and the findings from the following studies:
- A comparative analysis of organizational capability of the National Policy Framework (NPF) structures established to manage TIP, which contributed to learning and provided recommendations to strengthen the network;
- Review of the National Action Plan and Prevention and Combatting of Trafficking in Persons Act (PACOTIP) policies alongside the Government of South Africa (GOSA). This review involved stakeholder engagement, KIIs, observing training practice and other aspects of the criminal justice system pertaining to TIP reduction. It also reviewed existing TIP interventions by other development partners to determine existing gaps and opportunities for reforms or to strengthen practice and provided recommendations;
- Analysis of migration as a facet of TIP in South Africa and KII with Department of Labour provincial coordinators;
- Evaluation of the effectiveness of purposefully sampled programs within the National Freedom Network and independent organizations, using a case study methodology contributing to pathways for impact.
- A review of indicators and an emerging theory of change to determine pathways for impact
Impact:
- The Reports and our evidence and policy briefs are publicly available and have been shared widely on social media, and available on the USAID DEc
- The results of the study have been shared at numerous forums to provide an evidence base to inform future strategies, resource mobilization, prioritization and counter-trafficking efforts in South Africa by government, civil society and international organizations.
- This study led to a collaborative follow-on study, the Measures to Countering Trafficking in Persons (MCTIP).
Key Staff:
- Margaret Roper
- Heather Dixon
- Zamo Thwala
- Jesse Webb
Targeted Data Quality Review (DQR) of Global Fund HIV, TB and Malaria-Funded Programs in Somalia
Description:
Objective:
UNICEF Somalia commissioned Khulisa, with partner Daytam Research and Consulting, to conduct a targeted Data Quality Review (DQR) to assess the quality of reported data at public health facilities in Somalia. The DQR was conducted using standardized World Health Organization (WHO) DQR tools and processes on three indicators (HIV, TB, and Malaria) as well as two cohorts of the multidrug-resistant tuberculosis (MDR-TB) indicator (long and short regimen) reported in 2018.
The DQR aimed to determine the quality of HIV/AIDS, Malaria, and TB data reported in Somalia’s three regional areas: Puntland, Somaliland, and South & Central. Specifically, the DQR:
- Established a sample of 60 public health facilities and 24 national and regional area offices in the three regions;
- Assessed the quality of data for the selected indicators at the sampled facilities and higher reporting levels, including identifying gaps related to data availability, data flow, and data management systems structures and processes;
- Recommended improvements in data management; and
- Disseminated the DQR results for strategic planning and quality improvement.
Approach:
The DQR’s design was based on, and guided by, WHO DQR guidelines, which clearly outline assessment procedures at health facility and higher levels. The DQR was carried out in two parts:
- A desk review, which Khulisa conducted hand-in-hand with UNICEF and World Vision staff. Khulisa trained UNICEF’s antiretrovial treatment (ART) program managers and World Vision’s TB & MDR-TB program managers on how to use the desk review tools and supported these staff remotely until the tools were completed.
- A survey component, which Khulisa conducted in-person at the sampled sites. The survey component consisted of an assessment of the data management system and a physical verification of reported data against established data quality criteria.
Impact:
The evaluation’s conclusions, lessons learned, and recommendations informed UNICEF/Somalia and World Vision/Somalia, highlighting the need for the following improvements:
- Increase the coverage of facility visits and provide written feedback during each visit, including data quality issues;
- Document standard operating procedures (SOPs) and job descriptions for all staff with data management system responsibilities, including those doing data entry and report compilation;
- Clearly assign the responsibility of data entry and compilation of report to specific team members;
- Establish standardized data analysis and data use protocols, including data visualization, at facilities to enhance understanding of trends and to support planning at facilities;
- Promote and support a culture of data analysis and use from facility, to regional, to national levels, including the use of data for performance reviews and planning at all levels;
- Promote data reviews before reporting to the next level;
- Establish a data quality control procedure protocol;
- Promote a data quality control culture by training staff on data quality control at all levels; and
- Create an SOP around providing regular feedback, particularly on the data quality.
Key Staff:
- Kris Eale, Project Manager and M&E Expert
- Salome Omolo, Team Leader and DQR Expert
- Mary Pat Selvaggio, Project Director, and Quality Assurance
- David Ndou, Fieldwork, and data management support
- Adam Osman, In-country Research Supervisor
Tshivenḓa Benchmarking: USAID PERFORMANCE IDIQ Task Order2, Design Report F
Description:
Objective:
The U.S. Agency for International Development (USAID) Southern Africa, in consultation with South Africa’s Department of Basic Education (DBE), raised the need to conduct a study on setting early grade reading benchmarks in Tshivenḓa, one of South Africa’s 11 official languages.
Establishing reading benchmarks for Early Grade Reading (EGR) creates greater awareness of early milestones in reading development and helps teachers and schools ensure learners are reading at the required level. Under the PERFORMANCE IDIQ issued in 2018, Khulisa developed design reports outlining the approach, data collection and analysis procedures, and methods to establish African Language Benchmarks.
As a result of these initial reports, Khulisa received a task order in 2021 to collect data and develop reading benchmarks for Setswana (another official South African language) Home Language (HL) and English First Additional Language (EFAL) for Grade 1, Grade 3, and Grade 6. The DBE also commissioned reading benchmarks for the other South African languages. But Tshivenḓa, a Southern Bantu language mainly spoken by the Venda people in the northern part of South Africa's Limpopo province, is one of the South African languages that had not yet established early grade reading benchmarks. According to the 2011 census, 1.2 million people, or 2.2% of South Africa's population, speak Tshivenḓa as their first language.
The purpose of this study is to collect robust and quality data for analysis and establish reading benchmarks for Tshivenḓa Home Language learners in Grades 2, 3, and 6. The methodology will consider the fact that different benchmarking methods may be needed for Grade 2 and 3, and for Grade 6.
Approach:
Khulisa’s technical approach builds on our experience conducting earlier EGRS studies, the Reading Support Project (RSP) implementation evaluations, the data collection and analysis for setting benchmarks for Setswana HL and EFAL, and in producing reliable data while considering budget, time constraints, and other practical issues.
No EGR assessment tool had been developed for Tshivenḓa previously, so the Khulisa team factored in time for developing grade-specific EGR assessment tools, piloting, and analyzing the data for tool validation. We sampled 62 schools and assessed more than 7,200 learners. While the benchmark data does not require the use of any contextual data, we administered one survey for HL and EFAL teachers.
Impact:
The Tshivenḓa EGR benchmarks for Grades 2, 3, and 6 provide a benchmark against which learners can be monitored to check their progress and confirm if they are mastering the components of reading according to age, grade, and milestones. The benchmarks further provide a measure for schools, districts, provinces, and the DBE to measure the quality of education delivery.
Key Staff:
Margaret Roper
Zamo Thwala
Tshandapiwa Tshuma
Sithabile Ntaka
Measures for Countering Trafficking in Persons in South Africa (MCTIP)
Description:
Objective:
The MCTIP study aims to reduce the prevalence of TIP, as measured by the Prevalence Reduction Innovation Forum (PRIF) core indicators, and to reduce behavioral vulnerability outcomes among PRIF-identified TIP potential victims and survivors through the implementation of a randomized control trial (RCT) intervention. This RCT aims to assess if the proposed intervention model is effective in reducing the TIP prevalence among the target population. Additionally, the proposed work tests methods to systematically measure TIP and exploitation in high-risk populations. The intervention will include trauma-informed response planning and referral systems, developing and piloting the implementation of locally specific core standards of care (CSC), and providing tangible enhancements to mental health and substance abuse treatment and skills development to potential victims of trafficking. This forward-thinking approach could highlight the population prevalence of TIP and point to a reduction over time.
The objectives of the project, implemented in three phases over five years, are:
- Create a sample cohort of TIP potential victims for the RCT component (Phase I & II);
- Develop a clinically based and effective intervention program (Phase I & II);
- Demonstrate the evidence for intervention effectiveness (Phase II & III);
- Evolve the MCTIP into a sustainable and gradual TIP prevalence reduction intervention (Phase I-III); and
- Conduct monitoring, evaluation, research, and learning (MERL) activities (Phase I-III).
Approach:
The MCTIP study aims to contribute to South Africa's response by:
- Mapping Johannesburg and Cape Town according to TIP risk marker levels (Phase I);
- Collaboratively creating a localized CSC manual and response tool kits using a Delphi method (Phase I);
- Using the standardized PRIF screening questions and statistical rules to estimate the two sites' prevalence rates (Phase II & III);
- Testing the effectiveness of the CSC over 12 months, focusing on mental health, substance abuse, and skills development (Phase II & III);
- Strengthening the inter-sectoral and multi-stakeholder response.; and
- Provide training and capacity building sessions, dialogues, and forums with South African government departments and service providers to disseminate evidence-based knowledge on:
- (i) how effective the CSC are in ensuring potential victims are TIP-free (not meeting PRIF TIP criteria)
- (ii) which aspects of this intervention were successful or unsuccessful.
Intended Outcomes
- Collaboratively developed CSC manual localized to meet response needs, tried and tested for sustainable implementation and potential adoption in other South African cities.
- A proven and costed, trauma-informed approach that addresses the experiences of adults victims, and informs trauma responses that recognize life-long experiences and pathways toward healing.
- Contribute to the evidence base on what types of programs work best for whom, to reduce specific forms of human trafficking in a South African context.
- Develop a mapping methodology based on the risk marker level maps of Johannesburg and Cape Town, to provide visual toolkits for future monitoring and prevention efforts by South African and international response teams.
- Demonstrate a strengthened networking and collaborative approach and practice to strengthen the counter-trafficking sector.
Key Staff:
Margaret Roper
Thembi Mahlangu
Project Last Mile (PLM) Monitoring and Evaluation (M&E) Support for Strategic Marketing in South Africa: MINA Campaign Rapid Survey
Description:
Objective:
Project Last Mile (PLM) – an initiative of the Coca-Cola Company, the Coca-Cola Foundation, the U.S. President's Emergency Plan for AIDS Relief (PEPFAR), United States Agency for International Development (USAID), The Global Fund to Fight AIDS, Tuberculosis and Malaria (The Global Fund), and the Bill and Melinda Gates Foundation – aims to apply private sector best practices in strategic marketing and supply chain to improve the uptake and availability of health commodities and services across Africa.
In South Africa, PLM supports PEPFAR and the National Department of Health (NDoH) with a national behaviour change/marketing campaign aimed at men living with HIV entitled “MINA. For Men. For Health.”
In June 2020, PLM contracted Khulisa to carry out a rapid survey of the MINA campaign in 20 pilot clinics in Gauteng province, to take a “rapid temperature check” of the campaign’s pilot testing approach and future national rollout. The survey – carried out remotely due to COVID-19 – obtained feedback from 62 clinicians and PEPFAR District Support Partners (DSPs) in the 20 pilot clinics on the following aspects of the campaign:-
- Delivery and use of campaign collateral;
- Understanding of the campaign’s overarching objectives;
- Perceptions around the campaign’s strategic approach, materials, and message content; and
- Views on male patient reactions to, or engagement with, the campaign materials and messages.
The Khulisa study helped PLM to adapt the campaign’s implementation approach for greater effectiveness.
Approach:
Considering the COVID-19 pandemic, Khulisa’s solution involved remotely collecting data through telephone interviews.
The sample included clinic staff and implementing partner staff in the 20 pilot clinics located in Johannesburg and Ekurhuleni (two large metropolitan areas in Gauteng). Khulisa targeted a minimum of 60 interviews (all purposefully selected) based on contact information of on-boarded clinicians, clinic managers, and DSP staff.
- The Khulisa team presented the study’s findings in a PowerPoint report, as requested by PLM.
Impact:
Khulisa completed the study within seven weeks, interviewing 62 of 101 people identified. The team quickly generated results for PLM, both mid-stream and at the end of the survey, to enable modifications and adjustments to implementation of the pilot campaign.
Key Staff:
Mary Pat Selvaggio, Kris Eale, David Ndou, Chelsea Selvan
M&E Support to the Partnership for Aflatoxin Control in Africa (PACA), including a Baseline Evaluation of the PACA Secretariat
Description:
Objective:
The Partnership for Aflatoxin Control in Africa (PACA) commissioned Khulisa to conduct a wide range of M&E technical support services over a period of five years. Established in 2011 at the 7th Meeting of the Comprehensive Africa Agriculture Development Program (CAADP), PACA is an African Union (AU) initiative to protect crops, livestock, and people from the effects of aflatoxins – highly toxic substances associated with a wide range of severe health effects in both humans and animals. PACA is an innovative consortium that coordinates aflatoxin mitigation and management across the health, agriculture, and trade sectors in Africa.
To build government effectiveness and efficiency in tackling the aflatoxin challenge, PACA provides strategic leadership and catalyzes evidence-based, government-led actions at continental, regional, and country levels. PACA also fosters partnerships with Regional Economic Communities (RECs), such as the Common Market for Eastern and Southern Africa (COMESA) and the Economic Community of West African States (ECOWAS), the private sector, NGOs, and other development partners.
The spread of aflatoxins is directly linked to the regulation of imports and exports in AU member states. Thus, a key component of monitoring PACA is building capacity within member states for refinement of trade regulation and import/export supervision. Khulisa worked with relevant government ministries to collect and analyze data on trade patterns and inspections and to build capacity for monitoring and evaluation of aflatoxin control.
Approach:
Over a five-year period, Khulisa provided PACA with a wide range of M&E technical support, specifically:
- Refining the PACA Phase I theory of change and logic model;
- Developing the Phase I M&E Framework, including recommending key indicators;
- Finalizing PACA Phase II indicators;
- Generating PACA Phase II baseline data for the six pilot countries and PACA Secretariat;
- Developing tools for future routine M&E of the aflatoxin control in PACA-supported countries (based on the tools developed for the baseline data collection); and
- Scoping an M&E data/information storage and retrieval system for Phase II.
Khulisa also made research-based recommendations for an electronic M&E system for PACA to use going forward.
Khulisa completed/conducted the following activities:
- Baseline data collection for core indicators;
- Developed/reviewed/strengthened results frameworks and monitoring plans;
- Produced/reviewed/updated M&E operational manuals, guidelines, and tools;
- Developed/delivered training on results measurement; and
- Created IT systems for M&E
Khulisa also submitted draft and final deliverables, including reports with detailed findings and recommendations.
Outcome:
In undertaking this assignment, PACA sought to determine baseline values for 25 key performance indicators against which the results of PACA Phase II could/would be measured. During the course of this assignment, the Khulisa team and PACA worked collaboratively to unpack, reword, and refine the list of 25 indicators. The baseline results for the 25 indicators PACA selected for inclusion in the baseline evaluation were presented to the PACA Team (Secretariat and pilot country staff) at a specially scheduled presentation at the African Union Commission in March 2018.
Key Staf
Phase 1 – Mary Pat Selvaggio, Rukudzo Marapa, Jennifer Bisgard;
Phase II - Michael Ogawa (Project Manager), Edna Berhane (Team Leader), Assessors: Kris Eale, Puveshni Crozier, David Ndou
Evaluating the Effectiveness of Programs for Protection of Refugee Youth in Urban Settings in Africa
Description:
Objective:
In September 2021, the U.S. Department of State’s Bureau of Population, Refugees and Migration (PRM) contracted Khulisa to evaluate the specific characteristics of the urban youth refugee population – between the ages of 15 and 24 – in Egypt and Uganda. The evaluation sought to determine the main challenges refugee youth face and analyse the extent to which current PRM-funded activities are reaching this population.
The evaluation had three components. The first was a comprehensive desk review of 862 documents and written analysis of regional populations and programming by international organizations, such as UNHCR and UNICEF, and implementers of diverse programming funded by PRM. The second component was fieldwork to examine the situation on the ground in two African countries (Egypt and Uganda). The third component was the final evaluation report to inform PRM programmatic and diplomatic decision-making on humanitarian assistance.
Approach:
Khulisa used a mixed-method approach to combine the analysis of global administrative data with qualitative and quantitative data collected in Egypt and Uganda.
Khulisa conducted an extensive desk review, which served to understand the background of this group of refugees. The desk review also set the concepts and definitions used in the process of protection and relief of young urban refugees in Africa, which is supported by PRM, and informed the development of data collection instruments.
The evaluation team collected data from three groups of stakeholders: International organizations (IOs) receiving PRM funding, NGOs implementing programs with PRM funding, and refugee youth beneficiaries. Khulisa used a unique design to collect quantitative data, developing intercept interviews to survey 320 refugee youth at service delivery points in Cairo and Alexandria, Egypt, and Kampala, Uganda. The team collected qualitative data collected from 59 refugee youth, 23 NGO officers, and 11 IO officers.
Khulisa submitted a final report to PRM, which included comprehensive data analysis to answer the four research questions and presented 11 recommendations to PRM.
The evaluation team considered several elements within the scope of the evaluation, including:
- Diversity of geographic/demographic environment: PRM interventions take place in regions with large numbers of refugees spanning many cultures, languages, and environments.
- Diversity of sector and project focus: PRM supports multisectoral packages that cover a wide range of focus areas, including education, livelihoods, health, legal protection, and more.
- Diversity of project stakeholders: PRM relies on trusted partners such as UNHCR or UNICEF and other IOs and NGOs to implement its programs. These partners use diverse approaches to project implementation.
One challenge this evaluation faced was the difficulty accessing beneficiaries. Travelling to regions where refugees stay in urban areas is not always possible or safe. To overcome these obstacles:
- The Khulisa team carefully developed a socially sensitive, culturally sensitive, gender- and language-sensitive approach to ensure the data collected was as unbiased and truthful as possible. The evaluation team was complemented by local researchers and enumerators in both countries. We used local translators to provide the beneficiaries with a culturally sensitive environment to reflect on the services they were receiving and how PRM funding could better help them.
- The team travelled to NGO and IO service delivery points and intercepted beneficiaries as they were entering or exiting the site. The beneficiaries were safe on the premises of the service provider and had been briefed by the NGO/IO teams that our enumerators would be around. The survey was very short and designed to fit in easily with the beneficiaries’ schedules.
- At service delivery points, NGOs/IOs asked several refugee youth if they would be willing to participate in an in-depth interview with the Khulisa evaluators. We conducted 59 key informant interviews (KIIs) with refugee youth to provide in-depth, qualitative data on their experience.
- The evaluation team interviewed youth between the ages 15 and 18 by following specific protocols of inducing no harm and avoiding stress, while respecting young refugees’ rights and enforcing PRM policies. The team provided specific care to young refugees coming from war and conflict areas, as well as unaccompanied children.
To supplement the data coming from young refugees, the evaluation team also conducted KIIs with management and protection officers at UNHCR, UNICEF, and seven NGOs in Egypt and Uganda, as well as five focus groups with 58 mentors, coordinators, and implementers of the programs throughout the NGOs in Egypt. UNHCR in both Egypt and Uganda provided Khulisa with administrative data about refugee youth in each country.
Impact:
Khulisa submitted a thorough report to PRM and presented our findings and recommendations to the PRM Bureau. The final report included a triangulation analysis of all qualitative and quantitative data, as well as recommendations for IOs and implementing partners in the headquarters and in the field to improve the programming that PRM funds. The evaluation team identified case studies with best practices and impact on young urban refugees, which can be replicated in other countries and programs. The team also identified some diplomatic actions PRM can use with partner countries to engage governments on undertaking policy and other legal actions to improve the lives of refugee youth.
Key Staff:
Dr. Martha Muco,
Jesse Webb,
Dr. Monica Trevino
End of Programme Evaluation (EPE) for the Tuberculosis in the Mining Sector in Southern Africa (TIMS) Programme
Description:
Objective
The Wits Health Consortium (WHC) commissioned Khulisa to conduct an independent end-of-program evaluation (EPE) for its Global Fund-financed “Tuberculosis in the Mining Sector in Southern Africa (TIMS)” regional program. TIMS aimed to reduce the TB burden in the mining sector in 10 southern African countries.
The EPE’s overall purpose was to: (i) assess the TIMS grant’s achievements and effectiveness, as well as the adequacy of the program strategies toward reaching expected targets and results; (ii) identify lessons; and (iii) make recommendations to inform the next phase of the program.
The evaluation’s eight objectives were to:
- Assess TIMS progress from January 2016 to January 2018 against key outputs/milestones in the grant;
- Assess the effectiveness of strategies and intervention implementation and provide recommendations to further improve the project;
- Assess the strengths and weaknesses of partnerships with implementing countries, especially key ministries, mine worker associations, development partners, country coordinating mechanisms (CCMs), etc.;
- Assess program efficiency focusing on financial, programmatic, planning, implementation, and monitoring and reporting systems and processes;
- Assess the strengths and weaknesses of the management of the primary recipient vis-à-vis sub-recipients;
- Assess the strengths and weaknesses of oversight by the regional coordinating mechanism (RCM) to ensure the success of the regional program.
- Identify the opportunities and challenges that might affect the scale and sustainability of the program.
- Identify lessons learned and make recommendations to inform the next phase of the program.
Approach:
This evaluation demonstrates Khulisa’s capability to execute a rigorous evaluation of a complex, regional project covering multiple countries and stakeholders (i.e., implementers, governments, civil society, local authorities, and beneficiaries) at national, provincial, district, sub-district, facility, and community levels.
To answer the eight evaluation objectives, the Khulisa team used a non-experimental design that excluded the use of a comparison group, but which allowed for measurement of project trends and achievements against pre-defined project targets and objectives. The EPE’s main data collection methods were:
- Desk review and mining of programme data: Khulisa reviewed relevant grant documents and data to understand the evolution of priority areas and the development and continuum of program interventions and activities. The team also extracted program data from the WHC-TIMS Project Management Office (PMO) database and reports from January 2016 through December 2017, with a focus on the grant performance indicator data.
- On-site key Informant Interviews (KIIs): Through field visits in all 10 countries, our team conducted interviews with key informants (individuals and groups) to obtain perceptions and feedback around the design, implementation, and results of the TIMS program and its core interventions.
- On-site indicator validation: Khulisa conducted a mini data-quality assessment to validate data for the indicator, “Number of Key Populations (KPs) screened for TB”, reported by sub-recipients to WHC, and by WHC to the Global Fund.
Impact:
The evaluation’s conclusions, lessons learned, and recommendations were developed to inform phase II changes in the TIMS Grant. The findings and the recommendations highlighted the need for improvements in the following areas:
Principal Recipient Management and Coordination
- Clarified roles and responsibilities between the principal recipient, RCM, CCMs, and the Global Fund is crucial for maximizing TIMS and ensuring effective management of resources.
- Clear and systematic processes for setting agendas, conducting oversight, and disseminating key findings and next steps to all RCM members for improved coordination, enhanced transparency, strengthened accountability, efficient resource allocation, timely decision-making, and enhanced knowledge sharing.
- An online platform accessible to all sub-recipients and partners for enhanced internal communication and collaboration, improved planning and coordination, streamlined document management, real-time updates and notifications, increased accountability and transparency, and knowledge sharing and learning.
Internal Processes
- Sub-recipient selection: Adapt processes to evaluate changes since contract-signing, review financial documentation for sub-recipient selection, ensure proper procedures for activity selection by TIMS consortium members, and document any approved deviations from rules or regulations by relevant bodies.
- Financial efficiency: Formally define and measure the financial efficiency of each TIMS activity.
- Risk management: Ensure that the risk register is regularly/adequately reviewed and revised to identify all risks to the program at all times.
- Sub-recipient management: Implement timely funds disbursement, user-friendly financial reporting, efficient M&E and meeting structure, streamlined in-country activity management, internal communication strategies, midterm evaluation, and supply chain coordination.
Technical and Programmatic
- Health services management and coordination, including better training, more promotion of silicosis testing, improved coordination with laboratories, and improved collaboration with district health teams in each country.
- Engaging more female miners and local organizations focused on gender.
- Clarifying and unifying the regional TB screening tool to improve results of TB screening and active case management.
Key Staff:
Jennifer Peters, Senior Evaluation Expert and Team Leader
Puveshni Crozier, Public Health/TB Expert
Marianze Roux, Financial Management Expert
Mary Pat Selvaggio, Evaluation Manager
David Ndou, Jr. Evaluator
Kris Eale, M&E Expert
Salome Omolo, Public Health Expert
South African PEPFAR Partners’ Performance Assessment (SAPPPA)
Description:
Objective:
The SAPPPA project’s main goal was to provide the President’s Emergency Plan for AIDS Relief (PEPFAR) with an assessment tool to use in all its program areas, and to conduct objective and independent on-site performance assessments of PEPFAR partners.
Approach:
Khulisa and its consortium partners, Social & Scientific Systems Inc. (SSS) and Sustainability Solutions Africa (Pty) Ltd (SSA), conducted 377 performance audits (PAs) of 89 PEPFAR-funded implementing partners. The PAs supported program accountability by independently assessing the management and administration of PEPFAR programs. The PAs reported on partners’ adherence to PEPFAR and South African Government (SAG) policies and guidelines.
Khulisa was responsible for the following activities:
- Overall management of all PAs undertaken under this project, including regular meetings with the client and managing consortium partners;
- Recruitment, training, and deployment of assessors;
- Piloting of the SAPPPA tool and process;
- Regularly updating and revising the SAPPPA tool and process based on lessons learned from conducting the assessments; and
- Conducting the performance assessments of PEPFAR partners.
The Khulisa team developed 10 modules as part of the SAPPPA Performance Assessment Tool – eight technical area modules (one for each of PEPFAR’s program areas) and two cross-cutting modules (organization administration and fiscal management, and human capacity development and health systems strengthening). Each of the modules underwent various edits and versions based on feedback from stakeholders and pilot testing. The team tweaked the final version of the tool during the lifespan of the project but did not modify it significantly to allow for comparability of results.
Impact/Achievements:
Through the SAPPPA project, Khulisa and its partners:
- Developed a performance assessment tool (PAT);
- Conducted performance assessments, strengthening accountability;
- Developed key SAPPPA guidance documents, including process, implementation, and training manuals;
- Assessed management and administration practices;
- Enabled capacity development and systems strengthening;
- Examined organizational administration and fiscal management; and
- Recruited, trained, and deployed highly qualified assessors.
Key Staff:
| PROJECT MANAGEMENT TEAM | ||
| Doug Franke | SustainAbility Solutions Africa | SSA Project Manager |
| Edna Berhane | Khulisa | Project Coordinator and Khulisa Project Manager |
| Mary Pat Selvaggio | Khulisa | Project Director |
| Sue Griffey | Social Scientific Services | Tool Development Specialist and SSS Project Manager |
(1 per consortium partner), 32 Technical Assessors (both employees and consultants), and 15 admin/finance/contracts support staff
Schools2030
Description:
Objective:
Schools2030 is a new globally informed, locally rooted, 10-year longitudinal action research and learning improvement program. The program works with 1,000 pioneering pre-schools, primary schools, and secondary schools across 10 countries, searching for and supporting positive deviance about what works to improve holistic, quality learning for all.
The overarching objective for the Schools2030 program is to support countries in achieving the targets set for Sustainable Development Goal (SDG) 4: “Quality Education for All”. Schools2030 uses a bottom-up school improvement model, which empowers teachers to: assess their learners on locally meaningful domains; design solutions to improve learning; use assessments to track and iterate those solutions; and then showcase the solutions to allow for broader adoption and systemic change.
In 2022, Khulisa was commissioned to design an impact evaluation of the Schools2030 program and collect the baseline data for this evaluation. Our work is divided into three components:
- The Finalized Impact Evaluation Strategy determines the impact evaluation methodology for the remaining eight years of the Schools2030 program, including sampling strategies, establishment of a counter-factual set of schools for comparison, and finalizing the key performance indicators.
- The Baseline Evaluation validates the M&E tools, collects baseline data, and analyzes these data to provide a summary of the status quo at baseline, thus forming a useful comparison at the time of the mid- and endline evaluations. In addition, the baseline evaluation serves as a guide for the program implementation, identifying where challenges are arising and adjusting the strategy as necessary.
- The Endline Impact Evaluation aims to ascertain the effect the program had on the target audience in relation to the goals and objectives set at project inception, and the extent to which Schools2030 achieved its outcomes. In addition, this final evaluation will determine what works, what does not work, and the reasons why.
Approach:
Impact Evaluation Strategy
Khulisa is committed to focusing on participatory, collaborative, and social justice agendas, recognizing and enabling donors, governments, and communities to bring their own experiences, perspectives, and characteristics to the evaluation design, implementation, and usability of the results. Our core values include utilization and quality, with clients often commenting on how helpful and participatory our work is.
Baseline Study
The baseline study will follow the finalization of the impact evaluation strategy. The baseline study, as a component of the impact evaluation, uses a modified difference-in-difference design, in which the difference between the intervention group and the comparison group is measured at three points: baseline (2023), midline (2026), and endline (2029). Our team will collect data from school principals, teachers, and education stakeholders in each country, as well as from learners through learning assessments and classroom observations developed by the Aga Khan Foundation.
Expected outcomes of the study
The baseline study is expected to demonstrate the relevance of Schools2030 in each country context and how this program has empowered teachers to innovate solutions for their learners. It is expected that teachers and principals in the Schools2030 program have greater agency and engagement in national education dialogue than teachers and principals not in the Schools2030 program. This improvement is also expected to reflect in the learning outcomes achieved by their students. The study will be used to improve the implementation of the Schools2030 program in each country, influence a generation of similar programs in other countries, and measure the impact of the Schools2030 program over time
Key Staff:
Khulisa is mentioned as a global evaluation partner in the Schools2030 Annual Report: https://khulisa.com/wp-content/uploads/2023/03/Schools2030-Annual-Report-2022-1.pdf
Eswatini Client Management Information System
Description:
Objective
Khulisa has been part of a massive effort to upgrade health data systems for the Eswatini Ministry of Health (MOH). As part of the Eswatini Client Management Information Systems (CMIS) project, funded by the U.S. Agency for International Development (USAID), Khulisa’s local team has been involved in developing, rolling out, supporting, and maintaining the web-based CMIS electronic medical record system, which allows the Eswatini MOH to track health data from the national level all the way down to community and client levels.
Approach:
Khulisa was subcontracted by US-based prime contractor FEI Systems and then later with the Palladium Group (under the Data.FI project), to provide staffing services, technical software development expertise, M&E services, and operational backup for the CMIS.
Our team has augmented the unreliable microwave network connecting Eswatini health facilities with access point networks (APNs) – portable modems that use cellular data via one of the local cellular networks. The use of APNs has helped to stabilize and improve the reliability of the CMIS at health facilities.
The Khulisa team also worked to develop CMIS web-based dashboards to replace the previous, facility-based dashboards, which had to be individually updated at each facility. The new dashboard system synchronizes data from the CMIS with an open-source application called the District Health Information System (DHIS2). The DHIS2 dashboards will assist stakeholders to make informed decisions about resource allocation and disease prevention.
Impact:
Before the ECMIS project, the Eswatini MOH used a paper-based data management system. It was challenging for the MOH to track patients across different clinics and keep abreast of their treatment history. Now, when a patient goes into any health facility in Eswatini that is using the CMIS, the patient’s health history will be available to the facility worker.
Key Staff:
Michael Ogawa
Evaluation of the operationalization of the Global Alliance for Vaccines and Immunization (Gavi) strategy through Gavi’s policies, programmatic guidance, and use of funding levers.
Description:
The Global Alliance for Vaccines and Immunisation (Gavi) commissioned Euro Health Group (EHG), in partnership with Khulisa, to conduct an independent evaluation of the operationalization of Gavi’s strategy through policies, programmatic guidance, and use of funding levers. The evaluation assessed the degree to which Gavi policy and programmatic instruments are effectively operationalized to enhance Gavi’s contribution to delivering national immunization programme priorities.
The evaluation focused on: 1) identifying strengths and weaknesses in the strategy operationalization model; and 2) generating organization-level learning on Gavi’s strategy operationalization model.
Approach:
The evaluation’s design was built on a detailed theory of change focused on key Gavi support areas – such as vaccines, TCA, and health systems and immunization strengthening (HSIS) support – that served as the analytical framework for understanding and evaluating the operationalization of Gavi’s 5.0 strategy.
The evaluation team conducted structured document reviews and key informant interviews (KIIs). Khulisa staff and local consultants supported the core EHG team in collecting relevant information, conducting the KIIs, and writing country case studies across four countries in Africa.
Impact:
The evaluation’s conclusions, lessons learned, and recommendations are intended to inform ongoing changes to Gavi’s overall strategy operationalization model to improve access to new and underused vaccines for children living in the world's poorest countries.
Key Staff:
- Mary Pat Selvaggio
- David Ndou
Moshal Program M&E Support
Description:
Objective:
Since 2009, the Moshal Program has been dedicated to enabling individuals to transform their lives through higher education. Moshal is a social transformation organization that provides opportunities to promising individuals, financially supports their tertiary education, provides career development and psychosocial assistance, and more. Moshal collects data on their students’ academic progressions, psychosocial status, and intergenerational social mobility, and tracks their alumni’s career progressions and return on investment.
Khulisa helps the Moshal team make sense of their data and strategically think through which study areas, degrees, and institutions provide the best opportunities and have the best throughput rates for the scholars and alumni.
Approach:
Using the existing data management systems in place at Moshal, Khulisa conducts qualitative and quantitative analysis to provide succinct information on year-on-year development and the current status of students and alumni. Khulisa also uses Moshal survey data to create sector- and institution-specific analyses to aid senior management, as well as impact and return-on-investment studies to provide the team with more insight into their overall impact and standing.
Impact:
The Moshal Program uses the information Khulisa provides to make strategic decisions on which degrees to fund at which institutions, how to better support Moshal alumni, which modules require more tutoring support, how to assist their scholars and alumni with career and skills development, and more.
Key Staff:
Tamar Boddé-Kekana
USAID/West Africa Peace Through Evaluation, Learning and Adaptation (USAID PELA II)
Description:
Objective:
The purpose of the USAID PELA II is to enhance USAID’s effectiveness in preventing and countering violent extremism (P/CVE), peacebuilding, and strengthening democracy, human rights, and good governance (DRG) in West Africa, in line with the 2020-2025 Regional Development and Cooperation Strategy (RDCS) Development Objective 1: “Democracy, Peace and Stability Enhanced.”
USAID PELA II incorporates two sub-purposes.
Sub-Purpose 1: To enhance the acquisition of knowledge and information in order to improve the quality of decision-making on strategic and programmatic issues.
Sub-Purpose 2: To support USAID/West Africa to effectively utilize the knowledge gained through the project, which further supports the first sub-purpose of improving the quality of decision-making on strategic and programmatic issues.
Approach:
USAID PELA II supports USAID/West Africa’s role as thought leader through assessments, monitoring, and evaluation (M&E), analytical research, learning cycles, knowledge management, visual storytelling, and convenings. Recognizing the interrelationship between the USAID PELA II objectives and outputs, our implementation approach is based on a robust integrated learning solution to accomplish USAID/WA Regional Peace and Governance Office’s (RPGO) objectives on Regional Development Objective 1 (RDO-1).
Our iterative and continuous learning platform will integrate cross-cutting elements including Gender Integration and Social Inclusion (GESI), environmental and social impacts, conflict sensitivity and political awareness, climate change and resilience mainstreaming. These areas will help stakeholders identify knowledge gaps that impede effective and responsive programming in areas such as gender integration, youth involvement, community resilience, threats to stability, and drivers of violent extremism, as well as how these components are linked to each other, both incidentally and causatively.
USAID PELA II will also provide M&E, Collaborating, Learning and Adaptation (CLA) and communications support to USG Interagency Coastal West Africa programs under the U.S. Strategy to Prevent Conflict and Promote Stability (SPCPS) of the Global Fragility Act (GFA). Specifically, USAID PELA II will support regional and country convening, M&E, learning, adapting research and impactful storytelling to a broad range of stakeholders.
Team Khulisa brings extensive experience in WA alongside the thought leadership USAID PELA II requires to support the Mission’s appeal to achieve RDO-1.
Impact:
USAID PELA II will help stakeholders identify knowledge gaps that impede effective and responsive programming in areas such as gender integration, youth involvement, community resilience, threats to stability, and drivers of violent extremism, as well as how these components are linked. The provision of verifiable and credible information and analysis will boost stakeholders’ efforts to manage impediments to good governance, make decisions based on valid information, and respond more readily to citizens’ expressed needs and priorities. It will further enhance existing stakeholder relationships (bi-lateral, transnational, regional, local) through frequent engagement to inform decision-making.
Key Staff:
Accra team
Dr. Randal Thompson- Chief of Party (CoP)
Francis Ansong- Deputy CoP
Approximately eight technical and operational project staff
Home Office Team
Dr. Martha Muco, Project Director
Elizabeth Dwyer, Program Coordinator
Scaling Up Nutrition – Learning & Evaluation (SUN LE)
Description:
Objective:
Scaling Up Nutrition Learning and Evaluation (SUN LE), a USAID-funded Monitoring, Evaluation, and Learning (MEL) project, forms part of an integrated strategy to reduce childhood stunting – impaired growth and development of children due to poor nutrition, health, and care – in Zambia. Khulisa serves as the prime contractor for SUN LE in partnership with the Government of the Republic of Zambia and three institutional partners: Indaba Agricultural Policy Research Institute (IAPRI), ICF Macro, Inc. (ICF), and the University of North Carolina at Chapel Hill (UNC).
SUN LE directly supports the Zambian government’s Scaling Up Nutrition (SUN 2.0)/First 1000 Most Critical Days Program (MCDP) II with survey, research, evaluation, and dissemination services to facilitate learning, adaptive management, and improved accountability for achieving the goal of reducing childhood stunting.
Approach:
SUN LE supports the building of an evidence base SUN / MCDP program and policy decisions through conducting high-quality:
- Baseline, midline, and end-line population surveys of Zambian households,
- Demand-driven focused studies on a number of relevant topics, and
- Biennial SUN 2.0 performance audits.
Each of these use measurement methods and designs that comply with international standards.
SUN LE also encourages a culture of learning and adaptive management among Zambian stakeholders by supporting them to use SUN LE generated data and information. Khulisa convenes national nutrition conferences and annual district-level dissemination and data use workshops, and produces communication products, such as policy briefs, summary reports, and journal articles.
Impact:
SUN LE household surveys, focused studies, and performance assessments have contributed to an expanded evidence-base for both national and subnational SUN/MCDP program planning.
Through close collaboration with and support for the National Food and Nutrition Commission (NFNC) – Zambia’s coordinating body for SUN/MCDP II – SUN LE has helped strengthen its capacity to use and disseminate nutrition data for improved program implementation at all levels.
Key Staff:
Mr. Mathews Onyango, Chief of Party;
Ms. Patricia Sakala, Strategic Information Expert;
Mr. John Manda, Learning and Dissemination Advisor,
Ms. Edna Berhane, Quality Improvement Advisor,
Ms. Mary Pat Selvaggio, Project Director
Conducting Performance Audits for the Scaling Up Nutrition (SUN) Program in Zambia - Khulisa
Measuring the creation of “Green Jobs” in Kenya
Description:
In 2021 and 2022, Khulisa participated in a project measuring the creation of “green jobs”, or jobs that contribute toward an evolving green economy, in Kenya. The project, led by RTI International, included surveys of stakeholders in Kenya’s energy and higher education sectors to determine the current extent of green and “decent” jobs – jobs that pay a decent salary and offer good working conditions – in Kenya. The project also sought to determine the additional training and skills development needed to achieve a greener workforce in Kenya – supporting growing sectors like geothermal, hydro, solar, and wind power – in the future.
Khulisa developed and conducted a series of surveys in Kenya’s energy sector, collecting data from employees, CEOs, and human resources (HR) professionals in the industry. Khulisa developed the surveys in line with the definitions of green and decent jobs, aiming to help the higher education and energy sectors work together to create a competent green workforce. Khulisa surveyed 207 energy sector employees at various employment levels, 13 CEOs/specialists, and 49 HR professionals, using online surveys ranging from five to ten minutes long.
Before these surveys, very little was known about the market-relevant skills within Kenya’s green jobs sector, and how “green” and “decent” the jobs in the industry actually are. The project’s initial aim was to pinpoint discrepancies between what the higher education sector teaches students versus the knowledge and skills needed on the ground. However, due to the lack of consensus, this became a broader study into the discrepancies and difference in knowledge base between different job classifications within the green energy industry itself.
Survey results showed there is currently a significant gap between what tertiary education students learn and the actual skills needed to thrive as part of the green jobs workforce. Not only do students need to learn new and different skills, but current members of the green energy industry need to have access to upskilling and re-skilling programs that will give them the expertise they need to succeed in their jobs.
Key Staff:
Green Jobs: Working Toward a Greener Future in Kenya - Khulisa
Khulisa awarded USAID Education task order
Description:USAID/Southern Africa recently awarded Khulisa Management Services (Khulisa) the fourth task order under the Practical Education Research for Optimal Reading and Management: Analyze, Collaborate, Evaluate (PERFORMANCE) Indefinite Delivery Indefinite Quantity (IDIQ) contract. As mentioned in Bob Stross’ November 2020 NGO Financial Newsletter, this new multi-million dollar award entails Data Collection and Analysis for the Early Grade Reading Study (EGRS), the Reading Support Project (RSP) as well as Benchmarking.
Together with subcontractors Research on Socio-Economic Policy (ReSEP), Research Triangle Institute (RTI) International, and Benita Williams Evaluation (BWE), Khulisa will contribute to the body of research on early grade reading interventionsin South Africa’s North West province. Key activities include an impact evaluation that continues the longitudinal testing of Grade 6 learners to determine the sustainability interventions provided under the Early Grade Reading Study (EGRS I). Khulisa will also conduct an impact evaluationof the Reading Support Project (RSP) by assessing learners in Grades 2 and 3. Additionally, the team will contribute to early efforts of establishing reading benchmarks in English First Additional Language (EFAL) and Setswana Home Language (HL) by analyzing the impact evaluation learner results from the two sets of impact evaluation data.
Khulisa’s immediate aim is to kick off research on the social-emotional effect of COVID-19 on early grade reading learning and teaching. Survey service provider, GeoPoll, will perform computer-assisted telephone interviews (CATI) to remotely collect data from principals, teachers and parents/caregivers in relevant schools. Our team is thrilled to collaborate with USAID/Southern Africa in planning, designing, monitoring, evaluating and learning from interventions focused on improving the reading skills of primary grade learners in South Africa.
SANGAT II Evaluation
Description:From 2016 to 2019, DFID commissioned Khulisa to provide evaluation services to the Strengthening African Networks for Governance, Accountability, and Transparency Programme – Phase II (SANGAT II) and Twende Mbele (“Going Forward Together”). The three “networks” involved are the Asset Recovery Inter-agency Network Southern Africa (ARINSA), the Collaborative Africa Budget Reform Initiative (CABRI), and Twende Mbele, which is a network focused on Strengthening African [Government] Monitoring and Evaluation (M&E) programmes. Khulisa conducted a network analysis (2017), undertook a process evaluation (2018), provided an outcome evaluation of each network (from 2018 – 2019) and summarised the outcome evaluations in a final summary report (2019). Reports are available here: devtracker.fcdo.gov.uk
Khulisa strengthening health systems in Nigeria
Description:Khulisa provided support to Family Health International (FHI) 360 on the Strategic HIV/AIDS and TB Response Program (SHARP) in Nigeria. The program aimed to improve the quality and effectiveness of high-impact, evidence-based HIV/AIDS interventions to meet Nigeria’s goals and objectives at the state and local government area levels. Khulisa led the implementation of two operations research (OR) studies. The objective was to answer key operations research questions related to the improvement of service delivery and strengthening health systems with an expanded effort with the Government of Nigeria (GON) in Bayelsa, Edo and Lagos States
Khulisa conducts data quality review in Somalia
Description:UNICEF Somalia contracted Khulisa to conduct a targeted Data Quality Review (DQR) of four indicators (HIV, TB, Malaria and MDR-TB) reported in Somalia and Somaliland. We adapted the World Health Organization (WHO)’s DQR approach and tools to the local Somali context to assess data quality at 80 sites. Khulisa also assessed the three regions’ DHIS2 and provided recommendations on how to improve the quality of data captured on DHIS2. Through Action Planning Workshop sessions at the end of the DQR exercise, Khulisa presented the findings and recommendations to UNICEF, World Vision, and the Ministries of Health in Somalia and Somaliland, collected their inputs to strengthen recommendations, and established a Follow up Action Plan with proposed implementation timeline.
South Africa: Allan Gray Orbis Foundation External Evaluation
Description:Khulisa completed an External Evaluation of the Allan Gray Orbis Foundation’s Programs in South Africa in 2018-2019. The evaluation was conducted by a team from Khulisa that included two youth entrepreneurship specialists. During the inception phase, we conducted an evaluability assessment to determine the extent to which programs could be evaluated, after which we were able to propose a suitable design to the evaluation. The evaluation included: 1) design, including a review/revision of the Theory of Change 2) process and 3) outcomes/impact evaluation. To determine outcome/impact, Khulisa used a quasi-experimental design where we retrospectively followed program beneficiaries on their entrepreneurial development path to assess how they were supported by the programs. They were compared with a cohort of applicants who almost were accepted into the program. Khulisa followed a multi-stakeholder participatory approach. We conducted interviews with key Foundation stakeholders and used a combination of quantitative and qualitative data collection methods such as surveys, group discussions, and case studies to gather data from past and current program beneficiaries (Scholars, Candidate Fellows and Association members) and a comparison groups. The evaluation provided a number of strategic and programmatic recommendations to be taken forward by the Foundation in strengthening their programs.
Khulisa evaluated State’s CARSI Small Grants Program
Description:The U.S Department of State’s Bureau of Western Hemisphere Affairs’ Office of Policy Planning and Coordination, Strategy and Resources Unit (WHA/PPC/SRU) commissioned Khulisa to conduct a performance evaluation of its Central America Regional Security Initiative (CARSI) Small Grants Program (SGP) implemented in high crime neighborhoods across Belize, Costa Rica, and Panama between FY 2012-2015. Khulisa designed a participatory mixed methods research evaluation to determine the impact of the $15-million investment aiming to reduce crime and violence by targeting at-risk youth. Over a period of 18 months, Khulisa conducted an extensive review of program documents and literature; interviewed key stakeholders across the three countries; led in-person focus groups of SGP beneficiaries; and deployed two public opinion surveys targeting beneficiaries and the general population to gather perceptions about neighborhood safety, crime, and public service provision in the region.
Monitoring & Evaluation for International Development course
Description:Since 2016, Khulisa designed and has delivered a three-week Monitoring & Evaluation (M&E) for International Development course for graduate students at the Johns Hopkins University (JHU) School of Advanced International Studies (SAIS). The course leads students through a comprehensive overview and application of M&E and learning, including best practices, current challenges and trends in the field, and real-life examples. Through a mixture of lecture and practical application, students work in groups to design and present M&E plans based on case studies of past and current Khulisa M&E projects, and also performance indicators for these case studies. Luke Connolly, Associate Director, Internships & Experiential Learning at JHU SAIS, says this of Khulisa: “[They have] outstanding knowledge in their field; excellent and timely communications; personable employees; services are highly rated by students who consume their course.
Design evaluation for Reading Support Project (RSP)
Description:USAID Southern Africa contracted Khulisa to assist with planning, designing, monitoring, evaluating and learning from interventions focused on improving the reading skills of primary grade learners as part of the Practical Education Research for Optimal Reading and Management: Analyze, Collaborate, Evaluate (PERFORMANCE) activity. Under this contract, Khulisa conducted the Design and Implementation Evaluation of the Foundation for Professional Development (FPD) run Reading Support Project (RSP). Comprising 263 schools and over 1500 teachers, the RSP is focused on improving the reading teaching practices of Grade 1 to 3 teachers in Setswana and English First Additional Language (EFAL) in two districts in the North West Province of South Africa, with the ultimate outcome to improve learners’ reading skills. The first step in the evaluation was to assesses the feasibility of the RSP reaching its intended outcomes by looking at the program design, through a design evaluation. A key lesson from this design evaluation is that through developing process maps for this intervention, we (the evaluators) were able to identify potential barriers, challenges, and pathways to enable the achievement of the outcomes. It led to a number of recommendations for strengthening the design of the RSP. The report is available here. Khulisa conducted the implementation evaluation of the RSP, guided by findings from the design evaluation. Activities for this part of the evaluation included a review of RSP monitoring data and Data Quality Assessment (DQA), a fidelity review, a cost and time analysis, coaching case studies, and interviews and surveys with coaches, teachers, principals, School Management Team (SMT) members, district and provincial stakeholders, and other actors.
South Africa: Alumni Outcome Survey for PEP Academy program
Description:Contracted by Social Innovations, Khulisa conducted an alumni outcome survey for the PEP Academy program. The survey was conducted telephonically and through group discussions with a sample of youth who had participated in the program between 2009 and 2018. Findings informed the PEP Academy on the impact the program has had on their lives and provided program recommendations. The final report is available here: PEP Academy alumni survey 2019
Early Grade Reading Study Phase 1 & Phase 2
Description:To support the Government of South Africa, USAID Southern Africa is implementing the Practical Education Research for Optimal Reading and Management (PERFORM) program. The overall goal of PERFORM is to improve the reading skills of primary grade learners. Khulisa was awarded the Practical Education Research for Optimal Reading and Management: Analyze, Collaborate, Evaluate (PERFORMANCE) Indefinite Delivery Indefinite Quantity (IDIQ) contract to provide technical, analytical, advisory, monitoring, evaluation and related support services to assist USAID in effectively diagnosing needs, and planning, designing, monitoring, evaluating and learning from the PERFORM interventions. Under this IDIQ contract, Khulisa has performed data collection for the Early Grade Reading Study (EGRS) Phase 1 & Phase 2 in North West province. This Task Order entailed large scale data collection, collation, analysis, and reporting services to USAID and the South African Department of Basic Education (DBE).
Khulisa conducted oral and written early grade reading assessments with 9,656 learners. Khulisa also collected survey and observation data from 215 school principals, 631 teachers, and 3,462 parents. Khulisa collated, cleaned, and analyzed the data and supported the development of a technical report and learning brief. One of the unique features of the Task Order was the development and administration of a School Functionality Tool, which was used to collect 217 observations of school functionality. Full reports are available here: https://pdf.usaid.gov/pdf_docs/PA00X2VS.pdf
Evaluating antiretroviral treatment in South Africa
Description:The PEPFAR/South Africa Antiretroviral Treatment (ART) programme was transitioning towards an increase of South African Government ownership and stewardship, and Khulisa provided expertise to support the evaluation.
Khulisa’s ability to quickly mobilize technical experts, evaluators, analysts, and report writers enabled PEPFAR/South Africa to carry out a thorough and in-depth evaluation of the programme. USAID on numerous occasions praised Khulisa for its speed and responsiveness.
Read report
Evaluating bio-tourism portfolio in Mozambique
Description:Khulisa conducted a performance evaluation of three biodiversity and ecotourism activities in Mozambique: the Arco Norte, Lake Niassa and Gorongosa restoration projects.
The evaluation assessed the projects’ effectiveness, impact and sustainability to inform future USAID and/or Government of Mozambique biodiversity and tourism activities.
Read report
Supporting orphans and vulnerable children through education
Description:Khulisa evaluated the education support provided to Orphans and Vulnerable Children (OVC) in KwaZulu-Natal. The project was implemented by Research Triangle Institute (RTI International) in partnership with the KZN Department of Education, MiET Africa and JET Education Services. Read report
Western Cape Emergent Literacy Intervention (ELIT)/ Strengthening Teaching of Early Language and Literacy in Grade R (Stellar)
Description:USAID, in partnership with the Western Cape Education Department in South Africa, commissioned Khulisa to conduct an impact evaluation of the provincial Emergent Literacy(ELIT) project in Grade R under USAID’s School capacity & innovation program (SCIP). The project provided training and resources in emergent literacy, including an orientation to an early literacy programme called Stellar (Strengthening Teaching of Early Language and Literacy in Grade R) to all Grade R teachers in the Western Cape. Khulisa deployed a quasi-experimental research design in this provincial-wide study. The evaluation demonstrates our ability to design, execute and analyse robust impact evaluations. Read report
Evaluating an education Community of Practice (CoP) model
Description:The Zenex Foundation commissioned Khulisa to evaluate the effectiveness of a Community of Practice (CoP) model and methodology for a Mathematics and Science learner support community through the BRIDGE project. The evaluation aimed to explore whether BRIDGE’s model and methodology provided an innovative approach for developing sustainable CoPs, and whether participation in CoPs results in meaningful practice and policy change through to government level. Khulisa found that BRIDGE had maintained a high level of participation in their CoPs. Support from a wide range of stakeholders (NGOs, businesses, Foundations, CSIs, government, academia and individuals) attested to the value of participating in CoPs. This included the quality of dialogue and presentations, networking and sharing, quality information products as well as knowledge generation and management.
South Africa: Baseline assessment and end-term evaluation for FNB’s Primary Education Programme
Description:The FirstRand Foundation first entry into the primary education sector in South Africa was through the FNB Primary Education Programme (PEP). Tshikululu Social Investments, as the PEP programme stewards, commissioned Khulisa in 2012 to conduct a baseline school functionality assessment of 20 primary schools in two KwaZulu-Natal municipalities. This was followed by a further contract of 20 primary schools in the Free State province in 2013. The assessments identified current intervention projects that were taking place in the schools and their surrounding communities. Khulisa used 50 indicators of school effectiveness at primary school level to determine the school functionality baseline. In 2017, Khulisa evaluated the 37 programme schools of the 3-year PEP project against the baseline. The end-term evaluation findings was presented at the First Rand Foundation and Tshikululu Social Investments “CSI that Works” breakfast in October 2017 to education stakeholders, funders and government officials. Read report
CLEAR-AA Winter School
Description:Khulisa’s Jennifer Bisgard facilitated sessions on evaluation design for the foundation course and evaluability assessments for the advanced M&E course at the World Bank Center for Learning on Evaluation and Results (CLEAR-Anglophone Africa) as part of the Development Evaluation Training Programme in Africa, training officials from 22 African countries in August, 2017.
External Evaluation of the PEPFAR|USAID|South Africa-funded “Systems Strengthening for Better HIV/TB Outcomes” Project
Description:Khulisa conducted an external evaluation on USAID| South Africa’s “Systems Strengthening for Better HIV/TB Outcomes project” that was implemented by 7 District Support Partners (DSPs) working in 8 provinces across 21 districts of South Africa. The evaluation assessed the progress that the project made towards achieving pre-determined technical assistance objectives, anticipated outputs, and outcomes. It assessed the quality of the DSPs’ project implementation and determined which technical assistance approaches and activities worked and why. Findings of the evaluation were presented USAID| South Africa to inform the next phase of program design. Read report.
South Africa: Data Quality Assessments (DQAs) of selected PEPFAR-funded Orphans and Vulnerable Children and HIV Prevention Programmes.
Description:Khulisa conducted Data Quality Assessments (DQAs) of selected PEPFAR-funded Orphans and Vulnerable Children and HIV Prevention Programmes for USAID|South Africa, through FHI360-Capacity Development and Support (CDS). Khulisa assessed 5 health and social development indicators at 11 partners and 5 sub partners by visiting 91 NGO sites across seven provinces in South Africa. Using a capacity building approach, the DQAs assessed the strengths and weaknesses of partners’ data management systems and verified the quality of reported data to determine reporting error rates. DQA findings were presented to USAID|South Africa, FHI360-CDS, and assessed partners.
Evaluating elections training in Africa
Description:The Managing Democratic Elections in Africa (MDEA) Program is a regional elections capacity building project that trains African election officials on electoral management principles. Implemented by the University of South Africa (UNISA) in partnership with the Independent Electoral Commission, Khulisa’s evaluation found that the project was successful in bringing African election officials together and creating informal networks for learning. Read more
End of Project Evaluation of PSI’s project on Expanding and Improving Social Marketing in southern Africa Phase III
Description:Khulisa conducted an end-of-project evaluation of Population Services International’s Phase III project on Expanding and Improving Condom Social Marketing in southern Africa which aimed to reduce HIV incidence in Southern Africa. The evaluation focused on identifying lessons learned that could influence future Social Marketing programming in the southern Africa region. Khulisa undertook in-depth reviews of program documents, along with an on-line survey and Key Informant Interviews.
South Africa: Sustainable Use of Natural Resources Baseline Study
Description:For Earthlife Africa, Khulisa conducted a baseline study on the sustainable use of natural resources to improve climate change resilience amongst 350 women in eight South African communities. The project aims to uplift women in poverty by empowering them to manage and use natural resources, contribute to sustainable livelihoods, and improve their resilience to climate change as well as support training and capacity building of selected women in communities. The assessment covered the type of energy, water sources and farming solutions the women deployed in the Gauteng, Limpopo and Western Cape provinces.Khulisa employed a mixed methods approach eliciting information from a variety of different sources, which formed the basis for the impact assessment of the project at mid-term review and at the end of the project evaluation.
Empowering Local Radio Stations
Description:Khulisa evaluated the 3-year “Empowering Local Radio with ICTs” project in seven African countries for the United Nations Educational, Scientific, and Cultural Organization (UNESCO). The project aims to increase the quality of local radio broadcasting by integrating the use of ICTs, strengthening local reporting and engaging as beneficiaries, poor people’s participation in debates related to their social needs and livelihoods. Khulisa evaluated the project progress towards attainment of results at 32 radio stations across all seven countries, using a qualitative evaluation approach incorporating the OECD DAC criteria. For reporting, Khulisa has employed a unique evaluation approach – the Meta Story – to provide a birds-eye-view of the project’s performance.
EGPAF commissions Khulisa to assess Mozambican healthcare
Description:Khulisa conducted a data quality assessment (DQA) for the Glaser Paediatric AIDS Foundation (EGPAF) on 25 health indicators in 66 sites. The sites spanned health facilities in four Mozambican provinces. The DQAs helped verify the accuracy of data reported by the EGPAF sites. Khulisa provided debriefings with EGPAF and its local partner, Ariel and also assessed the respective data management systems in Maputo, Gaza, Nampula and Cabo Delgado provinces for the quality of data recorded on source documents as well as the reports generated and submitted to third parties.
Technical support to Global Fund Recipients and Country Coordination Mechanisms (CCMs)
Description:Between 2010 and 2017, as a subcontractor to MSH under the Grants Management Solutions (GMS) project, Khulisa carried out 33 assignments to provide technical support to Global Fund Recipients and Country Coordination Mechanisms (CCMs) in 16 countries. The technical support services provided by Khulisa involved strengthening and building the capacity of grantees and CCMs in grant M&E and oversight, programme management, and budget development.
Africa and India: Conducting Scoping and Field Research Studies for GALVmed
Description:In 2016, Khulisa conducted two scoping studies for GALVmed. The first scoping study led to a field study to collect data on household dynamics relating to livestock in Ethiopia, Tanzania and India. Khulisa used rapid group and household surveys to collect data from over 2800 male and female livestock-owning farmers. At minimal cost, the study helped our client better understand its beneficiary base. The second scoping study provided GALVmed with examples of businesses in the agri-input sector that are successfully selling their products to large numbers of smallholder farmers in Africa. In 2017, Khulisa conducted a scoping study that produced a proposal to collect quantitative data on vaccinators/sales agents, smallholder farmers and retailers in Uganda. The scoping study led to a field study that collected data from 670 male and female livestock-owning farmers, 57 retailers, and 53 vaccinators/sales agents. Khulisa was commissioned in late 2017 to provide support to GALVmed’s monitoring and evaluation activities by conducting various ad hoc writing assignments for data sharing and dissemination of findings.
- Household dynamics within livestock-owning smallholder households: India, Ethiopia, Tanzania
- Newcastle Disease Vaccine Adoption Cluster Studies: India, Nepal, & Uganda.
- Assessing the Economic Sustainability of early GALVmed Market Development Projects: 5 years on.
- Vaccinator Performance: India and Nepal
- Newcastle Disease Vaccine and Poultry Productivity Changes: Mayurbhanj, India
National Library of South Africa Country-wide Survey
Description:Khulisa carried out a national survey of public libraries in South Africa to inform the NLSA, the Department of Arts and Culture, Provincial and Municipal Library Services, as well as the Library and Information Services sector about the status of public access computing (PAC) and equipment, current capabilities, and future training needs in public libraries. The study is being used to prioritise investment funding, and ICT and training interventions, for public libraries in each province.
SASDC DQA and End of Term Program Evaluation
Description:The USAID-funded South Africa Supplier Diversity Council (SASDC) had a pattern of under-reporting by its corporate members. As a result, USAID could not measure the impact of its support to SASDC. Khulisa conducted a two part independent evaluation of SASDC to verify results reported to USAID, to explore drivers of under-reporting by corporate members, and to offer data management and programme value recommendations. The assessment recommended how SASDC and its partners could improve data management and data accuracy. The evaluation revealed a more positive story around SASDC’s results and highlighted several success stories around corporate partnerships with small and medium suppliers from disadvantaged backgrounds. This left SASDC (i.e. post-USAID funding) in a stronger position, in terms of their programme focus and data system strength.
Providing Strategic Project Management to SA Government
Description:Khulisa was appointed by GTAC to provide strategic project management of a group of technical advisors to support the turnaround of the Safety and Security Sector Education and Training Authority (SASSETA) through the South African Government Technical Advisory Centre (GTAC). Reporting directly to both GTAC and the SASSETA Administrator, Khulisa managed and coordinated the activities of a team of technical advisors who were appointed by GTAC to develop and institutionalize organizational systems, processes and procedures and create a staff performance management system that would enable SASSETA achieve its goals and mandate. The project supported the SASSETA’s shift towards unqualified audits
Africa: Creating a Theory of Change for Aflatoxin control
Description:The Partnership for Aflatoxin Control in Africa (PACA) is an African Union initiative to protect crops, livestock, and people from the effects of aflatoxins. Khulisa’s Theory of Change clarified the network’s implementation strategy.
Africa-wide Health Evaluation Support
Description:Khulisa’s Tabitha Kibuka full time M&E support to this USAID/Afr African Strategies for Health (ASH) project. Amongst various activities, Khulisa spearheaded the Health Strand at the 2014 AfrEA Conference, and edited and mentored the Special Health Edition of the African Evaluation Journal.
South Africa: Award-winning performance management information system
Description:Khulisa moved the South African Department of Science and Technology (DST) from a paper-based reporting system to a web-based reporting system. This initiative included work for all DST-supported science entities such as the South African Space Agency and the Southern African Large Telescope (SALT).
Evaluating early grade reading in Malawi
Description:Khulisa conducted a performance evaluation for USAID/Malawi Early Grade Reading Activity (EGRA). The evaluation identified capacity gaps to improve the reading skills of Malawian students in Standards 1-3 and identified the necessary course corrections to support the EGRA projects in meeting its goals.
Read More
Evaluation of USAID/South Africa Tuberculosis Program
Description:Khulisa used a participatory approach incorporating a Theory of Change model to evaluate USAID South Africa’s Tuberculosis (TB) Program. The analysed the programme’s performance at all project spheres, from South African government to community beneficiary level in nine provinces of South Africa with the aim of launching a follow-on project to support South Africa’s National TB Programme (NTP).
Read More
Evaluating HIV sexual prevention for USAID
Description:Khulisa conducted a mid-term performance evaluation for USAID’s Sexual HIV Prevention Program (SHIPP). The evaluation assisted the funder and its partners to improve the project’s future implementation.
Read More
Evaluating math progress in South African schools for Math Chairs
Description:Khulisa evaluated four Maths Education Chairs and two Numeracy University Chairs. Khulisa evaluated four Maths Education Chairs and Two Numeracy Chairs based in 3 provinces: Gauteng, Eastern Cape and Western Cape. Khulisa assessed the entire programme as well as the individual Chairs programmes at each university as they conducted research and implemented diverse education models to improve teaching and learning in Maths and Numeracy.
Using technology to empower local farmers for Farmer Voice Radio
Description:Khulisa built an M&E framework and conducted an intense process evaluation of the Farmer Voice Radio (FVR) project using local researchers and data collectors. The project convinced farmers to change their agriculture practices to achieve greater food security and improved livelihoods. Khulisa also developed 46 training tools and 15 supplementary training materials to support the implementation of the project.
Read More
Evaluating nutrition interventions for South African Presidency
Description:Khulisa conducted an evaluation of nutrition interventions for mothers and children from conception to age five. The evaluation assisted Government to identify inhibiting and facilitating factors that affect implementing scale-up of nutrition interventions. According to the South African Department of Health, the evaluation mobilised departments to design action plans for presentation to Cabinet.
Read More
South Africa: Building capacity of health promotion coordinators for Department of Health
Description:In 2014, Khulisa conducted a comprehensive baseline survey on South African health promoters across five districts in Gauteng for the South African Department of Health. The project recommended capacity building of current and incoming health promoters through orientation tools and knowledge promotion workshops. The 2014 project led to a second assignment in 2015, in which Khulisa developed materials and trained health promotion coordinators who adopted the Khulisa-designed training material across Gauteng
Evaluating the smallholder farmer sector for the South African Presidency
Description:Khulisa Management Services conducted a diagnostic evaluation of the government-supported smallholder farmer sector using theory-driven methods. The South African Presidency’s Department of Planning, Monitoring and Evaluation (DPME) is using this evaluation to inform policy around smallholder farmers.