Objective
The Elizabeth Glaser Paediatric AIDS Foundation (EGPAF) and its local affiliate, Fundação Ariel Glaser (Ariel), support prevention of mother-to-child transmission (PMTCT), HIV Care, and antiretroviral treatment (ART) services at numerous sites in Mozambique. Some of these sites are funded through performance-based financing (PBF) contracts.
EGPAF commissioned Khulisa to conduct a comprehensive external Data Quality Assessment (DQA) to confirm the quality of data reported on 21 PBF and 4 non-PBF indicators in four provinces of Mozambique. The DQA’s main objective was to identify data quality issues to resolve and strengthen the monitoring and evaluation (M&E) system. The DQA also assessed concordance (consistency) of data reports from PBF invoices and routine data collection systems.
Specifically, the DQA:
- Assessed the quality of reported data (using standard data quality criteria of validity, reliability, timeliness, precision, and integrity) in terms of its concordance with source registers submitted by EGPAF/Mozambique;
- Assessed concordance of data reports from PBF invoices and routine data collection systems;
- Estimated the margin of error of reported data vis-à-vis source data, by district and by province;
- Assessed the various data collection, processing, and management systems implemented by EGPAF/Mozambique;
- Assessed the flow of data from the initial point of data collection to higher levels of reporting;
- Identified areas of potential vulnerability that affect general credibility and usefulness of the data; and
- Recommended measures to address any identified weaknesses in the data submitted by EGPAF/Mozambique, as well as the M&E procedures and systems in place.
Approach:
Given that the quality of reported data is highly dependent on the underlying data management systems (DMS), Khulisa assessed and verified the DMS and data for EGPAF’s PMTCT and ART services in a sample of sites (service delivery points, data aggregation points, and head office operations). The DQA involved a desk review of project documentation and data and two procedures:
- An assessment of the DMS in five functional areas; and
- A physical verification of data reported in the defined reporting period against established data quality criteria.
Combined, these two procedures allowed Khulisa to a determine the adequacy of data management systems/processes, the quality of data produced within these systems/processes (against specified data quality criteria), and the adequacy of human and physical resources to produce quality data.
Both procedures include interviews with key staff involved with the DMS: M&E specialists/officers, nurses, data capturers, and people in charge of report development and submission.
Impact:
The DQA found that the EGPAF/Ariel data was valid and collected according to the definitions in the standard operating procedures. The main overarching recommendation highlighted the need for improvements in the following areas:
- Data Storage: Facilities should be provided with resources to protect registers and program data documents from being damaged by people or floods.
- Data Management: Establish a comprehensive data quality assurance plan that includes regular training for DMS staff at facilities to maintain accuracy and consistency in data handling.
- Data Management Personnel: Implement and regularly conduct mini workshops on standard operating procedures and guidelines dealing with data management.
- Technology: Introduce data capturing via a mobile device, in which information can be automatically uploaded to a cloud database, to mitigate the numerous shortcomings of the current data recording and collection system.