#EvalTuesdayTip: SAMEA Tips: Before Evaluating, Consider the “How, Who, What, and When” of Data Use

The theme of the BRIDGE M&E CoP session was, “Actioning educational data: Have data, what next?”. Leticia’s presentation focused on lessons learned from the Khulisa team’s recent Early Grade Reading evaluation in South Africa. Read more about the objectives of this evaluation.

Leticia’s presentation emphasized the importance of deciding how an evaluation’s data will be used before – not during or after – the evaluation takes place. When planning an evaluation, it’s important to consider the “How, Who, What, and When” of data use:

1. HOW will the data be used? 

In the Early Grade Reading evaluation, the data would be useful to inform policy and practice. Therefore, the evaluation team had to be intentional about how to collect the data to meet the client’s needs. For example, the South African Department of Basic Education (DBE) wanted to get immediate feedback on how COVID-19 had affected teachers and schools in order to provide targeted support to them. The client also wanted to ensure that the data emerging from the language benchmarks would influence teaching practice and district officials’ ability to support teachers in the teaching of reading. 

2. WHO will use the data?

The evaluation team knew there were different stakeholders who would benefit from the data. The DBE was a key one, but district officials, teachers, school management teams, parents, learners and other organizations working in the education space would also benefit from the findings.

3. WHAT does the data look like?

It’s important to think about which data collection methods will best provide the data you need. In this study we needed immediate feedback from teachers and principals during the pandemic, when COVID-19 was disrupting schools and we weren’t sure when school staff would be present in the workplace. Therefore we collected the data telephonically, through computer-assisted telephone interviews, and were able to provide the data to the client immediately after collection.

Another consideration is what the end product will look like, considering the different stakeholders who will use the data. In this instance we produced technical reports, summary reports, and learning briefs that would be accessible to a wide range of audiences. Other user-friendly data presentation methods include videos, infographics, and PowerPoint presentations. 

4. WHEN to deliver: Data must be timely.

Using telephonic interviews meant that the DBE received data at the right time to make decisions. There was a lot of interest in understanding the socio-emotional effects of COVID-19 quickly, so there could be targeted interventions to support teachers and schools. The evaluation team did not wait until the end of the 1.5-year evaluation to provide findings; the results were ready at the moment they were needed most. 

A couple of other data use tips:

• Data should be made available and shared widely whenever possible. For the Early Grade Reading evaluation, all reports and data are available on the USAID Development Experience Clearinghouse and the USAID Development Data Library.
• Other websites of interest include the DBE research repository, where many education reports are available.
• Opportunities for dialogue and collaboration encourage engagement with the data and therefore better data use. In this instance, we set up a roundtable with key stakeholders to discuss the findings from the COVID-19 research. We also presented the report findings to the client before sending them out for comment, which allowed us to clarify a lot about the data and findings at an earlier point in the process.