Study setting
Burkina Faso is a low-income country in Sub-Saharan Africa (SSA) that was ranked 182 out of 189 in the Human Development Index in 2019 (18). In 2017, the country’s maternal mortality ratio was 320 deaths/100,000 live births and in 2016 the neonatal mortality rate was 26/1,000 live births (19, 20).
The health system of Burkina Faso is a three-tier-system. In rural areas, primary health care is provided by the Centres de Santé et de Promotion Sociale (CSPS), which are usually staffed by nurses and midwives, as well as by community-level referral facilities in form of the Centres Médicales (CM), which in addition are also staffed by a physician or surgeon. Referral centers at the district and regional levels include district and regional hospitals (21). Especially remote areas, however, face a shortage of well-trained personnel (22). In addition, rural facilities often experience stock-outs of essential drugs and supplies due to inadequate procurement and distribution systems (23). Public (i.e. government-run) health facilities therefore rely on income from user fees for medical tests and drugs (24). In 2013, at the onset of our study, Burkina’s total health expenditure was 6.4% of GDP (25). Since 2002, ANC services have been user fee exempt (26), and since 2006 user fees for deliveries in a health facility have been reduced to 20% of the actual cost (27). Still, in 2010 only 33% of all pregnant women received the recommended number of four ANC visits, and only 66% gave birth in a clinical setting (28). In 2011, the Government launched a pilot PBF program in three districts to improve primary health care provision at district and community levels in an attempt to further reduce maternal and newborn mortality. This pilot PBF produced an increase in the number of ANC visits, institutional deliveries, and postnatal visits. In 2016 the nationwide user fee exemption policy (i.e. gratuité) was launched to remove the remaining financial barriers to maternal and child health services (29).
PBF design & implementation
As the results of the 2011 PBF pilot were promising with respect to primary care service provision, in 2014 the Ministry of Health (MoH) geographically expanded a revised PBF design with funding from the World Bank. This program’s main objective remained the improvement of the quantity and quality of maternal and child health services. In addition, the performance-based payments to health facilities were further combined with specific demand-side interventions targeting either the ultra-poor clients or the beneficiaries of a community-based health insurance scheme (30). The 2014 program was implemented in health facilities across six regions (Centre Nord, Centre Ouest, Nord, Sud Ouest, Boucle du Mouhoun, and Centre Est). In each region, the MoH identified two intervention districts based on their weaker performance with respect to key maternal health service outcomes. All CSPS and CM in these identified intervention districts participated in the PBF intervention. As part of PBF, health workers and facilities received additional performance-based payments for achievements measured by a set of quantity and quality indicators covering a wide range of primary care service provision including ANC. With respect to ANC, quantity indicators included the volume of services delivered to pregnant women attending ANC consultations and the number of pregnant women who received at least two tetanus vaccines during ANC. Quality indicators assessed the availability of key ANC equipment, supplies and drugs. Performance measured by quantity indicators was assessed monthly by external verifiers; performance measured by quality indicators was assessed quarterly by the respective District Health Management Teams. With each payment cycle, facilities received defined fee-for-service payments for the volume of services provided for each quantity indicator and an additional financial bonus based on their achievement score computed across all quality indicators.
Study rationale, design and sampling
The theory of change in PBF postulates that performance-based incentives in combination with financial decision-making autonomy enable health workers and facilities to increase the provision of targeted health services outputs (“will do”) by directly regulating the supply of structural and financial means required to achieve these improvements (“can do”) (11, 31). Consistent with existing literature on PBF mechanisms and related theory of change (11, 32), we therefore postulated that the PBF intervention in Burkina would produce positive changes in the availability of ANC equipment and drugs as well as an improvement in the adherence to ANC clinical protocols, such as key screening and preventive care. To assess the effect of PBF on the quality of ANC this study followed a quasi-experimental controlled design with two data collection points (baseline and endline) to compare changes in ANC service provision observed at primary level PBF facilities (i.e. CSPS and CM) between PBF intervention and control districts. For this purpose, two additional districts comparable in terms of health indicators and health system structures to those of the PBF districts were identified in each of the six regions. In these control districts, a random sample of CSPS and CM was surveyed. At each facility, a minimum of five non-randomly selected ANC visits on the day of data collection were directly observed.
Data collection
Data were collected at two time points: baseline (October 2013 – March 2014) and endline (April –June 2017). Trained data collection teams spent one day at each sampled facility to complete all survey questionnaires. For this study, we relied on data collected by two different questionnaires: (i) a facility assessment consisting of an inventory checklist collecting information on facilities’ infrastructural elements and health service inputs (including the number of ANC-trained staff, availability of ANC specific drugs or equipment, etc.); (ii) a direct ANC observation checklist completed for each observed ANC consultation collecting information on different clinical and non-clinical aspects of the provider-client interaction during a routine ANC visit, such as the assessment of the client`s current and obstetric history, physical examination, laboratory screening, or educational content. Clinical quality was measured against international ANC treatment standards.
Outcome and control variables
Our conceptual approach to framing and defining the quality of ANC outcomes is based on Donabedian’s framework on elements of quality of care (33). Our focus here is on clinical quality, i.e. inputs and processes related to effective ANC (34). Inputs and processes considered as effective ANC were identified from the WHO’s Service and Readiness Assessment (35); process elements were identified from a range of WHO recommendations and guidelines on ANC processes (5, 36, 37). Given recommended process elements relating to routine screening and prevention differ between clients attending their first vs. a follow-up ANC visit (37). We reflected this distinction by defining separate process outcome measures for first-time and follow-up visits.
The resulting five composite measures each reflecting different aspects of ANC quality are shown in Table 1, which were then populated with variables available in the two data collection checklists:
The first composite indicator, “service readiness”, combines key input elements required for quality ANC provision at the facility level, such as the availability of qualified clinical staff, equipment, and supplies to deliver quality ANC services. This indicator consisted of 11 variables taken from the facility inventory checklist.
The following two composite indicators, “screening of first visit clients” and “screening of follow-up visit clients”, combine process elements measuring screening activities at the case level, such as focused client assessment, and physical and laboratory screening for pregnancy-specific risk factors or complications. These indicators consisted of 24 (first visit) and 15 (follow-up) variables taken from the direct observation checklist.
The next two composite indicators, “prevention for first visit clients” and “prevention for follow-up visit clients”, combine process elements measuring prevention activities at the case level such as medical prophylaxis, client information and education on healthy behaviors during pregnancy, and birth planning. These indicators consisted of 11 (first visit) and 11 (follow-up) variables taken from the direct observation checklist.
All variables consisted of binary data. Composites were formed by additive aggregation of equally weighted variable items within each indicator. To allow for easier comparability, we transformed the values of each outcome indicator to a range from 0 to 1 in relation to their observed value range.
Table 1
Overview and definition of outcome indicators.
ANC outcome indicator
|
Analytical level
|
ANC quality aspects considered
|
Service readiness (11 items)
|
Facility
|
General readiness: general infrastructure (reliable water/electricity/communication/lab service), infection prevention (available sterilization device/ hand washing/ waste disposal)
ANC specific: ANC supplies (tests/drugs in stock), ANC staff/supplies (minimum staff, essential equipment)
|
Screening first visit cases (24 items)
|
Case
|
Focused history assessment: previous obstetric history (age, gestations, interruptions, premature death, stillbirth, previous hemorrhage, previous complicated deliveries), current obstetric history (provider asks about medication, LMP, bleeding, fever, headache/vision, edema, fatigue)
Focused physical assessment: vital parameters (provider checks weight, blood pressure, fetal heartbeat), physical parameters (provider checks conjunctives, edema, fetal size, presentation), diagnostic tests (RPR, HIV, Hb)
|
Screening follow-up cases (15 items)
|
Case
|
Focused history assessment: previous obstetric history (provider checks ANC card), current obstetric history (provider asks about medication, LMP, bleeding, fever, headache/vision, edema, fatigue)
Focused physical assessment: vital parameters (provider checks weight, blood pressure, fetal heartbeat), physical parameters (provider checks conjunctives, edema, fetal size, presentation)
|
Prevention first visit cases (11 items)
|
Case
|
Prevention: prescription/ information about folic acid/iron, information/ distribution of ITN)
Education: education pregnancy (diet, danger signs), education birth (education about emergency plan, SBA, birth, breastfeeding, family planning)
|
Prevention follow-up cases (11 items)
|
Case
|
Prevention: prescription/information about folic acid/iron, prescription of malaria treatment, provider explains importance of IPTp)
Education: education pregnancy (diet, danger signs), education birth (education about emergency plan, SBA, birth, breastfeeding, family planning)
|
ANC = antenatal care, Hb = hemoglobin, ITN = insecticide-treated bed nets, IPTp = intermittent preventive treatment of malaria in pregnancy, LMP = last menstrual period, RPR = rapid plasma reagin, SBA = skilled birth attended |
To model PBF effects on those composite indicators measured at the case-level, we identified a set of control variables we expected to independently affect the measured quality outcomes based on existing evidence in the literature. This included a provider’s training skills with respect to the adherence of clinical ANC protocols, suggesting that higher trained ANC providers positively influence the quality of ANC (38). In our study, we defined a categorical variable indicating whether a case was attended by an ANC provider with 3-year training (i.e., registered nurse, midwife, or physician), with 1-year training (i.e., auxiliary midwife or nurse), or without ANC training. Further, we included binary variables reflecting the relevant client demographics, including “parity” (i.e. nulliparous or not, “literacy” (i.e. ability to read and write vs. not), and “socio-economic status” (i.e. being from a household in the lowest wealth quintile vs. not), assuming lower parity, literacy, and higher socio-economic status to independently increase the likelihood of receiving higher quality ANC (38, 39). The definition and computation of wealth quintiles used in this study is described elsewhere (40). In addition, we defined a continuous variable “client age” in years and “consultation time” (i.e. duration of provider-client interaction in minutes), expecting older clients and longer consultation time to be associated with better quality ANC (39).
Analytical approach
To control for non-observed variables that potentially affected our outcome estimates, we treated the repeated facility measurements as longitudinal data. However, as ANC clinics took place on certain days in a week only, we had information on ANC case observations for only 67% of facilities in the baseline sample (but 94% of facilities at endline). For this study, we therefore decided to include only those facilities for which both baseline and endline data on ANC case observations were available.
We used descriptive statistics to compare sample sizes and characteristics over time. We used Pearson`s chi-squared test to identify differences in key characteristics for all sub-samples. To estimate the effect of PBF on the different ANC quality outcomes, we used a difference-in-differences approach based on linear regression. The assumption of parallel trends prior to intervention was confirmed using routine data from these study facilities for selected ANC indicators (41). As treatment assignment occurred at district level, we clustered standard errors at that level. In light of the small number of district clusters, we applied wild bootstrapping to further adjusted standard errors (42). Given the longitudinal nature of our data, we applied facility fixed effects to adjust the model for time-invariant facility characteristics. In estimating effect sizes for case-level outcomes, we further adjusted the models by the provider- and client-specific control variables outlined above. The resulting model specification is expressed by the following equation:
$${Y}_{dfit}={\alpha }_{f}+\beta {RBF}_{dfit}+\gamma {t}_{t}+\delta \left({RBF}_{dfit}\bullet {t}_{t}\right)+\varphi {X}_{it}+{\epsilon }_{dfit}$$
where Ydfit is the value of the composite score of each outcome variable for case i at facility f in district d at time t = 0 for baseline and t = 1 for endline; RBFdfit is a dummy variable which takes value 1 for a case i observed at a facility f empaneled under the RBF in district d at time point t and 0 otherwise; tt is a dummy variable indexing the time points of data collection (0 = baseline, 1 = endline); 𝑋𝑖𝑡 is the set of additional control variables in the case-level models; and εd𝑓𝑖𝑡 is the error term. Coefficient 𝛼𝑓 represents facility fixed effects, coefficient γ the time fixed effect, and coefficient 𝛿 the DiD estimate for the resulting effect size attributable to the PBF intervention.