The objective of the NSHIP was to expand services and improve quality of care in PHC facilities in three states in Nigeria and we find that this was indeed the case.
By the time of the survey in 2017, NSHIP intervention facilities offered a more comprehensive set of services and products across most of the dimensions measured, with PBF doing even more than DFF (Table 1). For example, PBF facilities offered an average of five more services, intervention facilities offered 1–2 additional lab diagnostics, DFF facilities stocked five additional general-purpose drugs (such as antibiotics), and PBF more than doubled the number of drugs, all compared to control. Immunization was the one exception to this finding and was provided in almost all facilities regardless of study arm.
We find that facility-level total revenues per capita are strongly associated with better performance, including offering more services, diagnostics, and products to their users. This is based on bivariate regression analyses (Table 2), in which revenue was significantly associated with outcome measures at the p < 0.01 level for all except immunization. The effect size shown in the regression coefficients for revenue was up to 0.281 for malaria services, equivalent to a 2.8% increase per one hundred Naira per capita increase in revenues at the mean. While the coefficients are not always large, they can still be meaningful. For example, looking at the total number of services offered, the coefficients are 0.11 for revenues, 0.12 for DFF, and 0.14 for PBF, which is equivalent to an increase of 4.9, 5.6, and 6.7 additional services respectively (See Additional File 1, Table A1.4).
Coefficients for DFF and PBF were also significant, which indicates that these facilities offered more services at a given level of revenue than control facilities did. From this, we conclude that both absolute funding levels and having the programmatic support to spend those Naira well were meaningful contributors to the observed NSHIP outcomes. In addition, the magnitude of the PBF coefficients are larger than those for DFF, even after accounting for higher PBF funding, implying that PBF increased services more overall, possibly due to the addition of an explicit incentive structure for certain behaviors. From these combined findings, we conclude that intervention facilities offered more comprehensive primary health care and were more prepared to provide high quality care, and that this was driven at least in part by increased funding available at facilities, in addition to the structured supportive supervision that they received.
The order in which services were added is also of interest, with the hypothesis that if financial resources were a barrier to offering care, then facilities would choose to offer simpler services that are less expensive to support. Indeed, the patterns of service offerings were generally consistent with this expectation, for example with more facilities offering oral contraceptives than implants (Fig. 2). Intervention facilities did not achieve fully comprehensive care, but they did outperform control across the board. The high rates of service offering amongst lower-revenue PBF facilities suggests that the explicit incentives to offer basic services were indeed effective in changing behavior. However, this conclusion is somewhat mitigated by the similar performance of DFF facilities with higher revenues, which did not receive direct incentives. This implies that the combination of adequate revenues with other aspects of the intervention package, such as supportive supervision and training, is enough to substantially increase service availability without the need for financial incentives.
In regressions to predict structural quality, the largest effect size is whether a service is offered or not (13 of 15 services), followed by being a PBF facility (9 of 15 services) and then DFF (5 of 15 services). The interaction terms are more frequently significant for DFF (5 of 15 services) than for PBF (Fig. 3). All of these coefficients are positive except for one (PBF HIV), which means that offering a service and being an NSHIP facility are positively associated with structural quality. Given that these outcome measures represent fairly complicated lists of equipment, staffing, and facility infrastructure, the positive response to NSHIP implies it may be due to the improved management quality that was supported by the intervention, especially early on in the PBF facilities.
In combination, PBF facilities have more structural quality overall but DFF facilities have more frequent interaction effects; this suggests that PBF facilities used their funding to increase overall facility readiness in accordance with the quarterly supervision checklist, while DFF facilities, which received half of the level of funding as PBF, had to be more careful in limiting their spending to only items on their top priority services list. This may be attributable to the difference in explicit incentives between the study arms.
This study has several limitations. Primary is the risk that there may be unobserved confounders that affect the results. This is particularly a concern for the control states, which could have had simultaneous policy or funding changes that affected their applicability as comparators. Additionally, we cannot account for changes in the vertical disease programs that may have happened in parallel. The second limitation is that we use the survey respondent’s statement that a service is (or not) offered in their facility as an accurate representation, although the response is not corroborated with documentation. This may introduce noise into the data, reducing the likelihood that we would detect a true underlying relationship. For example, immunization was “offered” in nearly all facilities at baseline, so even though stock levels increased during the study period, this was not reflected in our definition of “service offering”. Third, while we recognize its importance, we do not look at process quality due to data limitations. Last, we acknowledge that reported performance on a checklist is not the same as strong facility operations and there is the risk that facilities were “performing out” rather than making fundamental improvements, which has been documented elsewhere.31,32