2.1 Data Sources
Data for all analysis presented in this study were acquired from the Demographic Health Survey; a, a nationwide representative survey held every five years in developing countries (DHS Program, 2008). Responses of outcomes assessed were acquired from a verbal recall of caregivers, who were mainly mothers of infants.
Given that the financial reforms were mainly introduced in 2003, we selected four surveys, two before the policy introduction and other two after implementation as recommended by Leone, Cetorelli, Neal & Matthews (2016). To improve the level of analysis we selected surveys in the early 1990s to late 1990s and early 2000s as pre-intervention periods. Post-intervention surveys were selected between 2008 and 2018.
Nigeria was considered as the comparison country for this study as there was no clear federal level which targeted fees removal for child health services (Okonofua, Lambo, Okeibunor & Agholor, 2011). However, some states had piloted and implemented fee exemption for minimum packages for maternal and child health services. For example, the free maternal and child healthcare programme piloted in Enugu in 2007 (Ogbuabor & Onwujekwe, 2018). The Nigeria Demographic Health Survey (NDHS) data for 1990 and 2003 was used as the pre-intervention period whereas data 2008 and 2018 was used as comparative post-intervention period (DHS Program, 2008).
2.2 Outcomes under study
Eight outcomes pertaining to incidence and management of diarrhoea and fever were selected for this study; the incidence of fever (a proxy for malaria), medical treatment for diarrhoea, medical treatment for diarrhoea in a health facility owned by the government, given oral rehydration for treatment of diarrhoea, fever incidence, medical treatment for fever, care for fever in a health facility owned by the government and given antimalarial treatment. All outcomes were defined in line with the definition offered in the guide to the DHS statistics.
Diarrhoea and fever incidence had a binary response (Yes/No) as to whether any child under age five had any of the two illnesses two weeks preceding the date of survey (Rutstein & Rojas 2006)
Medical care for diarrhoea and fever is also a binary variable, and it was defined as the number of children with either illness, receiving medical advice from allopathic health sources irrespective of whether it is owned by a private or public entity (Rutstein & Rojas 2006). Those receiving medical care specifically from health facilities owned by the government was considered as an outcome as most financial reforms were initially implemented in those facilities recent expansion to the private sector.
Children receiving oral rehydration, a binary variable, was defined as children with diarrhoea two weeks preceding the survey which sought medical care and were given any form of oral rehydration as part of treatment. Likewise, those given antimalarial as part of medical treatment was defined as children with fever two weeks preceding the survey and sought medical treatment and received any type of antimalarial.
2.3 Statistical Analysis
To strengthen the quality of analysis, we combined propensity score matching (PSM) with difference in differences. Given that health financing reforms were nationwide, far reaching all significant parts of the health systems, we considered caregivers in Ghana as receiving the intervention and matched with untreated based on selected covariates in Nigeria.
Having first been developed by Rosenbaum & Rubin (1983), propensity scores predict the probability of receiving a treatment given selected covariates. To estimate propensity scores in this study, we selected a varied range of covariates from educational to socioeconomic backgrounds. Then, using a probit regression model, we predict the probability of intervention assignment to acquire the propensity scores.
To match intervention observations with untreated, we select a kernel matching technique, emphasizing on observations that fell within the area of common support. Kernel matching is preferred to one-on-one matching as it offers better matching controls (Berg, 2011). To reduce the possible bias emanating from the ex-post effect of the intervention, we match observations based on pre-intervention background characteristics.
Quality of post matching balance was assessed using mean differences between intervention arms and matching controls along with the percentage of bias, t test with p-value and variance ratios (Austin, 2009). As recommended by Rubin (2006), a substantial imbalance was flagged when the percentage of bias (via the mean difference) was above 0.1 and the variance ratio fell within the ranges of 0.8 and 1.25.
The second stage statistical analysis involved estimating the average treatment on the treated effects (ATT) using a difference in difference modelling. Pre-intervention trends were compared to post-intervention trends between the matched treatments and untreated. For pragmatic reasons of interpretation, a linear probability model instead of a logit or probit model was modelled to estimate impact. This was denoted as:
$${Y}_{i}=\alpha +\beta {T}_{i}+ \gamma {t}_{i}+\delta \left({T}_{i}*{t}_{i}\right)+{\epsilon }_{i}\dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots .\left(1\right)$$
Where α = the constant variable
β = specific effect ascribed to the intervention group
γ = time trend which is same between intervention and untreated groups
δ = the true effect, which is an interaction between the difference in outcome between treatment and untreated given the pre and post-intervention trends.
All analysis were conducted in STATA 13.0, specifically, the “psmatch2” package was used to create propensity scores and matching along with the “pstest” and “psgraph” to test the balancing property and graph results of the balancing respectively. The difference in differences was done by using the command “diff”. The sample size post matching was sufficient, therefore, no bootstrapping techniques was necessitated. Statistical significance was considered when p-value ≤ 0.05.