The findings of the study for both the countries, Pakistan, and Afghanistan, are presented under four thematic areas 1) Usability of the app: acceptability and operability, 2) Validity of data, 3) Health worker factors, and 4) Community response (table 3).
1. Usability of the app: acceptability and operability
Pakistan
All the LHWs and vaccinators who were interviewed at intervention sites used the Hayat application for the provision of immunization in the community. They found it useful and preferred it over manual methods because they can easily track children's immunization records at any time, citing ease of use and timely information sharing with supervisors. The app's user interface was generally satisfactory, encouraging its use in the field. However, some vaccinators faced challenges in understanding certain functions, suggesting the need for a more user-friendly interface in the local language. Nonetheless, minor issues were efficiently resolved by social mobilizers, highlighting their vital role in the rural health system. Overall, the app was well-accepted, improving immunization coverage and communication among health workers.
“It has made the contact easy, and we can timely talk to the supervisor, and it has brought ease in our work and if we face any trouble, we discuss it.” (FGD-Phandar)
Afghanistan
The Hayat application received positive feedback from all vaccinators and CHWs who appreciated its ability to enhance coordination with supervisors and streamline routine immunization processes. Younger CHWs and vaccinators expressed higher satisfaction with the app's features compared to their senior counterparts, who initially faced challenges due to limited digital proficiency. Respondents suggested the need for a user manual to aid in a better understanding of the app's functions. Additionally, internet speed and connectivity posed challenges in certain areas, prompting the participants to emphasize the importance of improved internet technology for timely reporting, despite the app's offline data entry option.
“Our new CHWs are well qualified and understand this application and they are satisfied. Old staff at times find it difficult to work with technology. The internet here is very slow and there should be a new technology through which the internet speed can be improved so that the vaccinators and the LHWs will be able to report timely” (FGD, Baharak)
2. Validity of data in monitoring and decision-making
Pakistan
The Hayat app's implementation in the district received positive feedback from EPI coordinators and managers, acknowledging its significant impact on planning immunization campaigns and resource allocation. The app's data proved valuable in monitoring and tracking resource needs, leading to improved district health metrics, and advocating for its extension to other districts. Participants in FGDs and key informants highlighted the benefits of digitizing manual registers, emphasizing their potential to reduce workload and standardize data collection across all districts. The digitized system was praised for eliminating fake data and ensuring a more active and accountable health workforce. The obtained field data was deemed valuable for policymaking, further supporting the recommendation to expand the app's usage to more areas.
“The data sharing process is improved especially the data of the LHW program. The actual data we get from the field helps in policymaking. We had recommended it be launched in the whole district. We recommended it for the other 18 UCs based on the improvement in work in the 6 UCs” (EPI coordinator, Gilgit)
District managers reported that the Hayat app significantly improved their ability to monitor and supervise community vaccinators and track their progress, which was especially valuable during the COVID-19 lockdown when accessing remote locations was challenging. Daily online data entry made it easy to track missing data and identify defaulters regularly, resulting in improved coverage of the target population. Vaccinators praised the app's features, such as providing a complete patient profile with contact information, which facilitated communication and organization.
“We get coverage data mostly from the vaccinators and it tells us the population of the children and through that information we see things, the EPI department tells us their target children and if they don’t meet that percentage then we check whether the issue is in field level or on the dashboard”. (LHW Provincial MIS coordinator)
Some district managers did not access the dashboard due to a shortage of time, thus hindering their ability to track the progress of the vaccinators in the field. They believed that regularly viewing the dashboard was inconvenient due to their various managerial responsibilities and increased workload. As a solution, they suggested integrating the app with the District Health Information Systems (DHIS) system of the health department.
“The app should be connected with the Management Information System (MIS) so that we will know vaccinator entries and will compare them with the hard copies if required. In addition to this, the Register system should be replaced by mobile-based data entry. We have 8 registers and all of them should be in the application and only one family register should be with us.” (District Coordinator EPI)
Afghanistan
The managers felt that the vaccinators were now more organized, and over time they noticed improved efficiency in their work. The problem of absenteeism has also been resolved to some extent. The FGD revealed that the overall facility-based deliveries were increasing because of awareness and guidance by CHWs. Reporting vaccination was never on time before but now with the help of this application, they can send work to supervisors promptly.
“The attendance has improved…before it was difficult to know the attendance even if a vaccinator is not on duty for a whole month. Now we call the vaccinators on the phone, and they tell us that they are busy with the outreach of the vaccination” (CHS, Shebar)
“The attendance has improved…before it was difficult to know the attendance even if a vaccinator is not on duty for a whole month. Facility-based delivery was also low but now, it has increased it is because of this application that community members have become aware of the complications of home-based delivery and they have developed an understanding of the ANC, and PNC services.” (FGD Shebar)
Additionally, EPI managers believed that the use of the app had resulted in improved data. District supervisors previously had to deal with the issue of zero doses and defaulters. The app’s built-in tracking features have made it easy to track zero-dose and defaulter children. The chances of missing the children's immunization have also been minimized.
“The zero dose and the defaulter coverage in the vaccinator data has improved and we are getting information about the dropouts.” (EPI focal person)
Health worker factors
Pakistan
Providing cell phone credit to LHWs had several benefits, including improved coordination with vaccinators and supervisors, and better communication with mothers for missed vaccinations. Incentivizing app usage and timely reporting was recommended by most vaccinators and LHWs to boost performance. However, initial data entry and picture-taking on the app increased their workload during the learning phase. Incentives for transportation were suggested to achieve full coverage, even though not directly related to the app. Health workers stressed the app's importance in communication with LHWs, LHS, and vaccinators. Some mentioned the need for incentives due to the extra effort in submitting reports, especially in remote areas with limited facilities. The app proved valuable in reaching pregnant women during the COVID-19 pandemic.
“This is a remote area and there is a lack of facilities. We also face transportation issues and that is why the LHWs think that they should be given incentives for the extra work” (LHS Phandar)
Afghanistan
The health workers expressed the need for incentives to support their work with the app in the community. Most CHWs and vaccinators felt that the app initially increased their workload during the learning phase of data entry. Supervisors recommended holding refresher training sessions twice a year to improve app proficiency for new staff and address their queries effectively. The importance of continuous training was emphasized, particularly for field workers with limited education and field experience, to ensure efficient app usage. These insights highlight the significance of providing proper support and training to optimize the app's effectiveness in community-based activities.
“This application has somehow increased our workload; despite this application, we have other responsibilities in the community as CHW. Now, it seems that our responsibilities have been doubled. We have to register for this application, which is reported to the AKHS office, and we have to also fill out other tele-sheets which is linked to the health post and the Ministry of Health. (Afghanistan FGD with CHW’s Baharak)”
Community’s Response to the App:
Pakistan
The in-built feature of awareness movies benefits LHWs by reducing session time and effort in community awareness work. Community workers and mothers accepted and appreciated antenatal awareness and immunization-related features, like videos and SMS alerts, for their engaging nature. Health workers suggested adding more health-related videos, especially for non-communicable diseases. The new app helps to elevate LHWs' position in the community and enables direct communication during labor and emergencies. LHWs regularly used phones to send post-delivery health education messages to women.
“A cell phone is useful in the delivery process because the women call the LHWs and also, we provide health education after the delivery. We should be given more health-related messages in it, regarding diabetes and heart problems. “(FGD, Phandar)
Afghanistan
Many mothers were reluctant to provide personal information and were concerned about data confidentiality due to security issues and the country’s unstable conditions. Also, some vaccinators and CHWs encountered threats and security problems when using the mobile application and entering data into it.
“This application asks for personal information which we then ask from the mothers of children during registration then they get offended they say, why do we ask them for their personal information every time? When we have to make an entry in the application, they say that we are not getting any benefits from this application so why are we asking them for so much personal information like telephone number and Tazkira (National Identity).” (FGD with CHW’s Baharak)
“There are also some security challenges involved in this application, the security situation of Baharak isn’t very good. As you know, household members often feel that we are using this application to record their voices. They feel that they may get into trouble because of this mobile-based application, and they discourage us from visiting their homes.” (FGD with CHW’s Baharak)