Table 1 shows the profile of the health facilities in the ten districts implementing the practice improvement approach before activities began. Data on the annual number of deliveries were obtained from the HMIS.
Table 1: Baseline (2015) profile of health facilities in ten districts implementing the practice improvement approach
District
|
Number of
health centers
|
Number of
hospitals
|
Number of
deliveries in 2015
|
Phase 1 districts
|
Rwamagana
|
14
|
1
|
9,187
|
Ngoma
|
12
|
1
|
9,478
|
Kamonyi
|
13
|
1
|
7,029
|
Musanze
|
15
|
1
|
12,410
|
Phase 2 districts
|
Huye
|
16
|
1
|
10,239
|
Nyamagabe
|
19
|
2
|
8,805
|
Nyaruguru
|
16
|
1
|
6,247
|
Gatsibo
|
19
|
2
|
14,825
|
Nyagatare
|
20
|
1
|
14,941
|
Nyabihu
|
16
|
1
|
8,405
|
Total
|
160
|
12
|
101,566
|
Source: Rwanda HMIS
Input level variables: Service readiness
At baseline in 2015, 35% of health centers and 67% of hospitals had newborn resuscitation equipment (bag and mask) and at endline in 2018, 98% of health centers and 100% of hospitals had at least one bag and mask (newborn size) at the facility. During implementation of the practice improvement approach 993 (51%) of the estimated 1,960 eligible[1] providers in the maternity and neonatology units in the 172 health facilities were recruited for clinical mentorship and given an initial assessment in HBB, 95% (943/993) of whom were nurses and 5% (50/993) of whom were midwives. Over three quarters (78%, or 772/993) began the mentorship process, 712 (92%) of whom also received LDHF.
Output level variables: Improvements in provider knowledge and skills
The average pre-test skills and knowledge score, conducted before the initial LDHF training session for providers, was 44% and the average post-test score was 88%. The clinical knowledge and skills assessment applied at the end of the project included 64 providers randomly selected from the 172 health facilities in the initiative. These providers scored an average of 85% on HBB skills and knowledge using the same assessment tool used during mentorship. It should be noted that the providers for the endline assessment were selected among all providers, whether they had directly received offsite mentorship or not, as all providers at least received peer-to-peer mentorship.
Figure 3 shows the percentage of assessed providers who scored 80% or above at each mentorship visit. The majority (60%) of mentees passed (i.e., received a score of 80% or above) on the first mentorship visit; 86% by the fourth visit; and 100% by the sixth visit. In other words, after one mentorship visit, 60% of mentees were competent. An average of 9% more mentees passed with each mentorship visit until 100% of assessed mentees had. After obtaining a passing score, a subset of 220 learners were followed for an average of five months. Nearly all (98%) of the participants maintained or improved their skills.
Outcome variable: Improvement in clinical practice
Figure 4 shows the percentage of live newborns not breathing at birth successfully resuscitated. The percentage increased from 70% in April- June of 2016 to 91% in Oct- Dec of 2018. There is a notable rise in the success of resuscitations reported from the last quarter of 2017 to the first quarter of 2018. This coincided with an intensive data quality intervention. After supervisors had repeatedly noted that there was inconsistency among providers in whether they reported tactile stimulation as the first step in resuscitation, a decision was made that providers would only report those resuscitations in which they used bag and mask. This guidance did not change the definition of the indicator in HMIS, but rather clarified it, and was disseminated to all districts and providers across the country. This emphasis on correct reporting resulted in a nearly 50% drop in the number of resuscitations reported, and a 17% rise on the proportion of successful resuscitations.
Improvements in health outcomes
Three of the four relevant facility level newborn health outcomes tracked in the HMIS improved between 2015 and 2018, and the improvement in the fourth indicator just missed statistical significance (Table 2). Neonatal admissions due to asphyxia showed the largest improvement, from 22.0 neonatal admissions due to asphyxia per 1000 live births in 2015 to 13.9 in 2018. This was an annual reduction of 12.9% (95%CI = 8.3-17.5%). Neonatal deaths due to asphyxia at health facilities decreased from 3.8 deaths per 1000 live births to 3.3 deaths per 1000 live births, representing an annual reduction of 4.2% (95%CI = 0.2% - 6.3%). The number of fresh stillbirths per 1000 deliveries decreased from 10.2 in 2015 to 7.5 in 2018, a 6.2% annual reduction (95%CI = 2.1% - 11.4%). Death within 30 minutes of birth, although not a global standard, is routinely reported in the HMIS. It is meant to capture deaths that are most likely due to asphyxia. Death within 30 minutes of birth decreased from 3.2 deaths per 1000 live births to 2.6 deaths per 1000 live births, a 4.0% annual reduction. This just missed statistical significance, with the 95%CI crossing zero (95%CI = +1.2% to -8.0%), which was not statistically significantly different from no decrease (p=0.06).
[1] Eligible providers include all clinical staff in the maternity and neonatology units in the 172 MCSP supported facilities. This includes an estimated 1,960 providers.
Table 2. Newborn health outcomes at health facilities in the ten implementation districts, 2015-2018
|
Year
|
Annual improvement, absolute and percent
|
95% Confidence Interval
|
|
2015
|
2016
|
2017
|
2018
|
Neonatal admissions to the Neonatal Intensive Care Unit due to asphyxia per 1000 live births
|
22.0
|
20.9
|
13.4
|
13.9
|
-2.8
(12.9%)
|
-1.8 to -3.8
(8.3% - 17.5%)
|
Neonatal deaths at the health facility due to asphyxia per 1000 live births
|
3.8
|
3.2
|
2.9
|
3.3
|
-0.2
(4.2%)
|
-0.01 to -0.3
(0.2% - 6.3%)
|
Fresh stillbirth at the health facility per 1000 deliveries
|
10.2
|
9.7
|
9.4
|
7.5
|
-0.6
(6.2%)
|
-0.2 to -1.1
(2.1% - 11.4%)
|
Newborn Deaths within 30 minutes of birth at health facilities per 1000 deliveries
|
3.2
|
2.8
|
3.0
|
2.6
|
-0.1
(4.0%)
|
+0.03 to 0.2*
(+1.2% to -8.0%)
|
*p=0.06
Provider perspectives on mentorship
Interviewed providers expressed satisfaction with the mentorship approach, stating that they felt more confident to provide care to clients after receiving support from mentors. Respondents noted that patient flow and logistics within the maternity ward also improved, contributing to effective triage and management of emergency cases. One midwife noted that after her hospital received training and mentorship, critical cases in the maternity ward were managed in a designated space with dedicated staff.
‘Before we had no place reserved for maternal emergencies. We did not give much considerations to those emergencies. They shared the same wards with other cases. After training and mentorship, we created a room for critical cases, and we allocated staff to take care of them. The triage is done and the critical cases are placed in that room for special care.’ – Hospital midwife
Clinical staff felt that newborn resuscitation skills improved after mentoring was implemented. One provider noted that newborns in distress frequently died because clinicians did not correctly apply the resuscitation approach. With the ability to practice and receive real-time advice from a mentor, health providers felt that newborn resuscitation was being employed correctly and was saving the lives of newborns who might not have previously survived.
‘Before, a baby with fetal distress died easily because we lacked knowledge about how to help the baby to breathe or how to do resuscitation. We were all ignorant in the matter of APGAR scores, and resuscitation was done without following resuscitation steps, but today, no child dies due to birth asphyxia. We try our best; we resuscitate; we call for help; and we save lives.’ - Health Center Midwife
Data use for decision making is another area that providers felt had improved after they received support and guidance. Relevant service statistics and quality of care measures were not routinely collected before the intervention was implemented. One staff member noted that her colleagues appreciated the value of quality data and the key role it can play in decision making at the facility level.
‘Before it was very hard to get actual data. Staff didn’t know why they needed to collect all these data. You would ask for the number of deliveries done and receive data which was not related to reality, simply because they didn’t know the importance of reporting good data. After receiving capacity building and data management… changes are remarkable, everyone knows how to collect good data and its importance in decision making.’ - Hospital Maternity Matron
‘Before submitting a certain report, I check and ensure the quality of my data. One time we found that the number of partograms was not equal to the number of normal deliveries. There were missing partograms, we discovered that the missing partograms were attached to transfer notes. We made copies of the missing partograms and showed the issue to the staff.’ - Health Center Midwife
‘I have been a tutulaire in different health centers for more than 16 years, we’ve had several trainings in various domains, topics or subject but once you left the training facility it was finished, no one asked you what you studied so that you share it. And most times you already even forgot about it, but with the LDHF approach, there is a difference; the training was happening onsite, more staff benefited in terms of theory and practice as it was done on site. Cases would be managed even during the training.’
- Health Center Titulaire