Objective:
In September 2021, the U.S. Department of State’s Bureau of Population, Refugees and Migration (PRM) contracted Khulisa to evaluate the specific characteristics of the urban youth refugee population – between the ages of 15 and 24 – in Egypt and Uganda. The evaluation sought to determine the main challenges refugee youth face and analyse the extent to which current PRM-funded activities are reaching this population.
The evaluation had three components. The first was a comprehensive desk review of 862 documents and written analysis of regional populations and programming by international organizations, such as UNHCR and UNICEF, and implementers of diverse programming funded by PRM. The second component was fieldwork to examine the situation on the ground in two African countries (Egypt and Uganda). The third component was the final evaluation report to inform PRM programmatic and diplomatic decision-making on humanitarian assistance.
Approach:
Khulisa used a mixed-method approach to combine the analysis of global administrative data with qualitative and quantitative data collected in Egypt and Uganda.
Khulisa conducted an extensive desk review, which served to understand the background of this group of refugees. The desk review also set the concepts and definitions used in the process of protection and relief of young urban refugees in Africa, which is supported by PRM, and informed the development of data collection instruments.
The evaluation team collected data from three groups of stakeholders: International organizations (IOs) receiving PRM funding, NGOs implementing programs with PRM funding, and refugee youth beneficiaries. Khulisa used a unique design to collect quantitative data, developing intercept interviews to survey 320 refugee youth at service delivery points in Cairo and Alexandria, Egypt, and Kampala, Uganda. The team collected qualitative data collected from 59 refugee youth, 23 NGO officers, and 11 IO officers.
Khulisa submitted a final report to PRM, which included comprehensive data analysis to answer the four research questions and presented 11 recommendations to PRM.
The evaluation team considered several elements within the scope of the evaluation, including:
- Diversity of geographic/demographic environment: PRM interventions take place in regions with large numbers of refugees spanning many cultures, languages, and environments.
- Diversity of sector and project focus: PRM supports multisectoral packages that cover a wide range of focus areas, including education, livelihoods, health, legal protection, and more.
- Diversity of project stakeholders: PRM relies on trusted partners such as UNHCR or UNICEF and other IOs and NGOs to implement its programs. These partners use diverse approaches to project implementation.
One challenge this evaluation faced was the difficulty accessing beneficiaries. Travelling to regions where refugees stay in urban areas is not always possible or safe. To overcome these obstacles:
- The Khulisa team carefully developed a socially sensitive, culturally sensitive, gender- and language-sensitive approach to ensure the data collected was as unbiased and truthful as possible. The evaluation team was complemented by local researchers and enumerators in both countries. We used local translators to provide the beneficiaries with a culturally sensitive environment to reflect on the services they were receiving and how PRM funding could better help them.
- The team travelled to NGO and IO service delivery points and intercepted beneficiaries as they were entering or exiting the site. The beneficiaries were safe on the premises of the service provider and had been briefed by the NGO/IO teams that our enumerators would be around. The survey was very short and designed to fit in easily with the beneficiaries’ schedules.
- At service delivery points, NGOs/IOs asked several refugee youth if they would be willing to participate in an in-depth interview with the Khulisa evaluators. We conducted 59 key informant interviews (KIIs) with refugee youth to provide in-depth, qualitative data on their experience.
- The evaluation team interviewed youth between the ages 15 and 18 by following specific protocols of inducing no harm and avoiding stress, while respecting young refugees’ rights and enforcing PRM policies. The team provided specific care to young refugees coming from war and conflict areas, as well as unaccompanied children.
To supplement the data coming from young refugees, the evaluation team also conducted KIIs with management and protection officers at UNHCR, UNICEF, and seven NGOs in Egypt and Uganda, as well as five focus groups with 58 mentors, coordinators, and implementers of the programs throughout the NGOs in Egypt. UNHCR in both Egypt and Uganda provided Khulisa with administrative data about refugee youth in each country.
Impact:
Khulisa submitted a thorough report to PRM and presented our findings and recommendations to the PRM Bureau. The final report included a triangulation analysis of all qualitative and quantitative data, as well as recommendations for IOs and implementing partners in the headquarters and in the field to improve the programming that PRM funds. The evaluation team identified case studies with best practices and impact on young urban refugees, which can be replicated in other countries and programs. The team also identified some diplomatic actions PRM can use with partner countries to engage governments on undertaking policy and other legal actions to improve the lives of refugee youth.