Table 3 shows the demographic information for the 530 PLHIV enrolled in the full trial, the subset of the 104 PLHIV who responded to the HITUES survey and the subset of 25 PLHIV who participated in the in-depth interviews. The mean age ranged between 34 and 36 years, and across the three groups the majority of participants (~60%) were women. Most were in a monogamous marriage, and initiated ART at WHO clinical HIV stage 1. The majority reported having access to a mobile phone at baseline (88%-100%), slightly fewer reported having access to mobile money at baseline. By the end of the study, 81% of participants had registered their mobile money account with the mHealth system.
Acceptability
Fingerprint Scanning: Patient Perspective
We found that most PHIV were comfortable with the fingerprint scanning and the mHealth system overall, especially given the growing ubiquity of biometric identification in Tanzania. Several mentioned that biometrics are the way things are moving in the country, and that many businesses are already using it.
The good thing [about the fingerprinting] is about confirmation, because if the fingerprint is not yours then the system won’t confirm…. I think fingerprints are clearer therefore service provision becomes easier… I knew it was something useful in most sectors that’s why I didn’t see any problem or feel bad.
Some PLHIV felt that fingerprint scanning added not only accuracy in confirming their identity, but also legitimacy, security, and enhanced reliability in tracking visits and medication pick up.
The current system which we used to scan should continue…this system is assured…the data remain safe and in a good system. By the previous system, you can forget, you can also lose the papers; different from the current system where information is kept electronically.
Participants noted that the fingerprint system simplified the appointment process and remarked that they spent less time at the clinic once they started using the fingerprinting and mHealth system. While the mHealth system did not completely obviate the need for paper files, the perception among some was that the system did streamline the clinic process and procedures. However, in contrast to those who talked about the efficiency or simplicity of the system, some participants brought up that they felt the system caused delays at the clinic because of its dependency on a reliable network connection, and delays related to staff who were not proficient with the system.
What I can say is sometimes when you come [to the clinic] and meet one person. When you come another time you find a new person. So, some pharmacists were good and some I can say didn’t know how to use it [mHealth system], so if you meet with the one who is well experienced you don’t spend a lot of time but another might tell you the system is not working because they don’t know how to use it.
Mobile Money: Patient Perspective
Using the survey and enrollment data, we found high levels of acceptability of the mobile money system and the fingerprint registration process among PLHIV. Nearly all (99%) of those who had access to a mobile bank account consented to automatic cash disbursement through mobile money. Further, we found that 98% of eligible participants consented for the study (consenting included fingerprint scanning).
In both IDIs and the structured survey we asked participants (in the cash groups) about their experiences receiving the cash, and regardless of how they received the cash during the study, whether they would in general prefer to receive the transfers in cash or delivered through mobile money. The interviews point to considerable variation of preferences for delivery in cash as compared to delivery as mobile money, and considerable variation in the rationale cited for these preferences. Specifically, one theme that emerged related to the acceptability of mobile money included accuracy, safety, and reliability. For example, those preferring the cash transfer delivery via mobile money noted that this method was more secure and more accurate:
I can say there is accuracy because when money is sent from the machine to my phone it means there is a report that will be sent. Therefore, there is no security in giving someone cash in hand because the sender may not get the money to the intended person or they might. So, there are two possibilities but the system is more accurate.
A second theme that was discussed related to control over spending and ability to save money. Interestingly, several PLHIV talked about how they used delivery via mobile money as a way to save or control spending; they were less likely to simply spend the cash on the way home if the money was in their phone versus in their pocket, for example.
…another thing is when you have the money in hand you may end up spending all of it in things that are of no importance., But when it’s in the phone you can leave the clinic …and the money remains in your phone. When you get a problem you withdraw it but if you have it in hand you get tempted to buy random things that aren’t important. So keeping it there helps, it’s like a small bank, your personal bank.
A third theme that was brought up by PLHIV related to privacy concerns. Some noted that mobile money offered increased privacy compared with receiving cash; in particular, mobile money was discreet enough that their participation in the study could remain private and was not revealed to others attending the clinic, for example. However, others noted that they felt that the cash was more private that the mobile money option – some expressed concern that someone would see the message on the phone that appears when the money has been delivered and know they received money. Related to this, some women noted that cash allowed for more decision-making power – for example they may share a phone with their partner, and if the money is delivered through the phone, their partner will see the SMS message, and they will no longer be able to decide how to spend the money on their own. In addition, others noted that they were concerned that the SMS message alerting them that the mobile money had been delivered might reveal something about their HIV status or that they were participating in a study.
I: Why you like to receive [the cash] at hand?
R: Because I live with my husband there at home and he is not supportive to children, sometime my children may need a small amount of money to use at school but he will not help. Sometime they need books, so it helps me but if sent through the phone, he must know it when he read the messages in my phone and he will start to question and this will be a problem.
Other themes that emerged related to preference for cash over mobile money included concerns about the technology infrastructure (e.g., the unreliable nature of the network) and fees related to the use of mobile money services (see supplemental tables for sample quotes).
Clinical Staff Perspective
To understand the level of acceptability of the mHealth system among clinical staff, we evaluated themes emerging from the in-depth interviews with clinicians, pharmacists and pharmacy staff. We defined acceptability from the clinical perspective as the degree to which the mHealth components are considered reasonable and satisfactory given current clinical working environment. One of the themes that emerged in discussing the mHealth system with the clinical staff related to the perception that of the system into the clinics would result in additional work for the staff.
…on the side of staff, it [mHealth system] will be something new which has been added to them, it will be a new task which used not to be there, this is what I see will be like a challenge to workers because they will see it as an extra work, it can bring like a sort of resistance to change but with time they will cope it.
A second theme that emerged was related to training on the system – some noted that the system was easy to use once they had adequate training, that it simplified their work, and helped with managing patient flow within the clinic.
…at first, I didn’t know how to use this system but now I can use it very well. At first, I was afraid to use this system and I told them that I can’t work with it and they told me that you will know it, so they directed me how to use it and I can now use it.
Finally, a theme related to system fatigue emerged in our discussions with the clinical staff. Some clinicians noted that many systems come and go as part of research or government programs, and the clinics often do not see any lasting benefit.
R: Because these systems are always present…and other new systems will keep coming, because there are even some new systems that we did not find here before, but the kept coming and we captured them and we are still working with them till now and they are good, so even if this one comes officially then I hope that it will be captured by the servants and they will just understand.
Appropriateness
Patient Perspective
We used the HITUES to assess the appropriateness of the automated mobile money system and the fingerprint components of the mHealth system among PLHIV (Table 4). We adapted the HITUES to explore domains of appropriateness, defined here as the perceived fit of the mHealth components (mobile money, fingerprinting) within the existing clinical care context for PLHIV. Those four domains included impact, usefulness, ease of use, and user control (see supplementary materials for full question text). The overall average score for the scale as it related to using fingerprinting for biometric identification was 4.1 (out of a total possible 5.0). Scores for the fingerprinting showed even less variation than those for the automatic mobile money disbursement, ranging from 4.0 (usefulness domain) to 4.2 (user control domain).
The overall average score for the scale as it related to the automatic mobile money disbursement (compared to receiving cash in hand) was 4.2 out of a possible 5, indicating that participants found the system both acceptable and useful, with minimal variation by domain. The impact domain had the highest average score (4.3 out of a possible 5 points).
Clinical Staff Perspective
We explored appropriateness of the mHealth system with clinical staff through the in-depth interviews. We defined appropriateness as the perceived fit of the mHealth system components within the existing clinic context. Discussions converged around the following themes: technical difficulties, staff shortages and staff turnover, facilitation of patient follow-up, benefits for the patients, and spillover effects for patients not enrolled in the study.
In terms of technical difficulties, some staff expressed frustration, as the system did not always work, and was dependent on having a reliable network connection.
Maybe what gives me a challenge is your system of research, for example a customer may come, you will pass him/her through the system, if the customer put his/her finger prints, the system fails to show recognition, you may try the left hand but it does not respond, the same with the right hand also, so you have to use the ID number, so this is what challenges me.
Other clinicians discussed the difficulties of implementing the systems when there are substantial staff shortages and staff turnover. Such shortages meant that job duties and roles changed regularly, as managers shifted staff around to cover gaps.
My opinions are, first let me start with the positive ones, the admission of the system went hand in hand with offering of benefits to the clients and it even helped us in reducing the number of lost follow-up clients meaning on those days that a client is on appointment, he does not show up, though it had its challenges.
Additionally, others noted that the system helped with reminders for when patients were due for viral load testing, and some pointed out the potential for the system to help with patient follow-up across clinics if the system were to be implemented nationally or even regionally.
My feeling is that [with the system], we will first reduce LTF [loss to follow-up] and there will be frequent tests because if a patient scan his/her details will automatically come as the way they are and I think that a person can test himself or herself in another place, If he/she has already being recruited in this system, I think if he/she just scan, the details will appear automatically, so we will reduce LTF because most of the LTF appeared because of repetition of testing, you may find that he/she was taking medicine from another place and then he/she come to our place and find it difficult to follow his/her transfer or he/she has lost the card so he/she decide to do new registration. I think re-registration of patients will end.
Several clinicians talked about how the mHealth system benefitted patients during the study, and discussed how the mHealth system had the potential to benefit all patients in the clinic were the system to be implemented broadly. Specifically, they noted that the system helped patients transition from every month prescription pick-ups to 3-month prescription pick-ups as it helped with visit attendance and thus with adherence.
Others recognized that the system helped patients come to appointments on schedule and remarked that the system led to more complete viral load testing as it helped to reduce loss to follow up. Finally, some clinicians recognized that the system had positive spillover effects even for those who were not enrolled in the study.
…because the system is there and there is a close follow up, it helps to remind us that the patient is required to conduct a test, it has motivated us and increase our attention in making follow up, not only for those who are in the system but also for all patients in general, it has helped us improve the service in the HVL [HIV viral load testing] in general.
Adoption
Patient Perspective
To measure adoption, we explored the proportion of PLHIV study participants who had their mobile bank accounts linked to the mHealth system at the end of the study. At the end of the study, 88% of study participants reported that they had access to a mobile phone, 78% reported that they had access to a mobile money account, and among those in the cash groups, 81% had registered their mobile money account with the mHealth system.
Clinical Staff Perspective
We measured adoption from the clinical staff perspective by examining the proportion of time that the pharmacist was operating the mHealth system – the intended design. As the mHealth system was rolled out in the clinics, research assistants provided significant levels of support to the pharmacist in registering patients with the mHealth system, however, by the end of the study, the pharmacist was operating the system on average 72% of the time (as opposed to the research assistant being the primary operator).
Fidelity
Patient Perspective
Among PLHIV in the cash award groups (n=346), 331 (96%) received at least one cash transfer during the 6-month study, and the average number of cash transfers per study participant was 4.7 (out of a possible 6 transfers). Out of a total of 1,651 cash transfers delivered to study participants, 1,283 (78%) were delivered through mobile money (the remainder were paid to participants in cash) and 2.5% of those sent through mobile money required manual re-sending due to network failures.
Clinical Staff Perspective
To assess fidelity of implementation from the clinical staff perspective, we looked at the proportion of visits for which the mHealth system was used. Over 3,067 total clinical visits across all four health facilities during the 6 months following study enrollment, 172 (5.6%) were not captured in the mHealth system; 94.4% of all visits were registered into the mHealth system at the time of visit. The proportion of visits for which the mHealth system was used ranged from 94.6% to 89.1% by clinical site.
Feasibility – Clinical Staff Perspective
To assess feasibility, we focused on the technological aspects of the mHealth system implementation, exploring how frequently the fingerprint scanning system and mobile money distribution systems failed. The clinic pharmacy structured observations captured data on 2,293 patient visits over the course of the study. Observations were focused on the use of the mHealth system during clinic visits and included the number of fingerprint scans required until the mHealth system successfully identified the patient, and whether or not the fingerprint scan was eventually successful in identifying the patient (regardless of the number of scans required). It should be noted that as more PLHIV enrolled in the study and more fingerprints were added to the database, finding correct fingerprint matches became more complex, and required several iterations of the matching algorithm. Overall, fingerprint recognition succeeded for 74.1% of visits while 25.9% required manual entry of the patient's unique identification number due to poor image quality. The success rate for fingerprint recognition increased over time; by the final month of structured observations, the success rate was 87.3% (Figure 1). Overall, the average number of fingerprint scans required for the mHealth system to successfully identify the patient was 2.04; this also varied considerably by study month, and by the end of the study, the average number of scans was 1.8.
Sustainability – Clinical Staff Perspective
In an effort to explore scale-up and sustainability, we asked the clinicians about what challenges they might anticipate encountering should the mHealth system offset some paper-based aspects of the current recordkeeping system. Many of the themes that emerged had already been mentioned, including the importance of training, considerations related to staff turnover, and the need for a salary top-up as many staff saw use of the system as an added task rather than as a way to simplify or facilitate their existing work.
The potential benefit for patients was another theme that arose when discussing sustainability of the mHealth system. Several clinicians noted recognized the potential benefits and mentioned that with adequate training, they would be prepared for the new system.
We are positively ready for it [mHealth system] as I have said that we have observed its high impact within this short time of using it. So, for us we have received it with all our hands and we wish if it would come earlier, it would be more better because data shows, data shows us that it has good results so we are well prepared about this, and if there will be changes which will make us to change our flow pattern or service provisions in general we are positive about it because we have observed the positive impacts so as the clinic we are ready for the changes which will come.