The selection procedure of articles is shown in Fig. 1. From the 242 identified articles, 17 full-text articles were subjected to eligibility assessment. Six eligible articles from 4 countries were included for the quality assessment (20–25).
As listed in Table 1, the included studies evaluated the reliability, construct, content, and criterion-related validity. Mostly, the studies provided the construct, content and criterion-related validity. Based on the classification of kappa score using the Landis and Koch criteria, the application of the the Maternal Data Extraction Tool (M-DET) in the study of VA in India conducted by Montgomery in 2011 was found to have the highest kappa score (κ = 0.85, 105 maternal deaths), however for their study conducted in 2012 with greater sample size, the value of kappa score was catagorized into substantial (κ = 0.66, 1040 maternal deaths) (22, 23).
The study of reported M-DET was aimed to assign not only the underlying cause of maternal death but also to analysis the interrater-reliability of contributing conditions including: (i) antenatal care accesss and final pregnancy outcome; (ii) planned place of birth and care provider; and (iii) consultation, transport, hospital admission, referral and number of health care contacts. The agreement in the assignment of contributing conditions of maternal death in this M-DET VA was varied. They reported almost perfect agreement for determination of gestational age, pregnancy outcome, transport, death en route and admission to hospital as the contributing factors of maternal death. To determine of the cause of maternal mortality in the M-DET study, the ICD-10 cause of death was used to classify the obstetric cause of death (22).
ICD-MM, the only tool used for facility-based MDR in this review, focused on the contributory causes of maternal mortality related to clinical aspects. The ICD-MM was developed to solve the misinterpretation in determining the underlying (primary) and contributing (secondary) factors of maternal deaths (26). The study of Owolabi in 2014 and Mgwadere in 2016 reported there was fair agreement between the facility-based review team and an expert panel using the ICD-MM classification (κ = 0.219, 53 maternal deaths and κ = 0.37, 86 maternal deaths, respectively). Because of the diverse variability of contributing factors, the levels of agreement among the groups cannot be accurately calculated (24, 25).
Table 1
Systematic review of included articles
First Author,
Year
|
Settings
|
Sample size
|
Type of maternal audit
|
Electronic instruments
|
Identify contributing conditions of maternal death
|
Validity
|
Standard of WHO coding of maternal death
|
κ
|
Construct
|
Content
|
Criterion
|
Comparator
Instrument
|
Fottrell et al., 2007
|
Burkina Faso
|
258
|
Verbal autopsy
|
Yes
INTERVA-M
|
-
|
+
|
+
|
+
|
Origin physician review
|
-
|
κ = 0.80
|
Midhet, 2008
|
Pakistan
|
110
|
Verbal autopsy
|
No
|
-
|
-
|
+
|
-
|
Record assigned by doctor
|
-
|
κ = 0.378
|
Montgomery et al., 2011
|
India
|
105
|
Verbal autopsy
|
No
M-DET (Maternal Data Extraction Tool)
|
(i) antenatal care accesss and final pregnancy outcome; (ii) planned place of birth and care provider; and (iii) consultation, transport, hospital admission, referral and number of health care contacts.
|
+
|
+
|
+
|
Trained physician vs midwife
|
ICD-10
|
κ = 0.85
|
Montgomery et al., 2012
|
India
|
1040
|
Verbal autopsy
|
No
M-DET (Maternal Data Extraction Tool)
|
-
|
+
|
+
|
+
|
Independent trained physician review
|
ICD-10
|
κ = 0.66
|
Owolabi et al., 2014
|
Malawi
|
53
|
Facility-based MDR
|
No
MDR
International Classification of Disease Maternal Mortality (ICD-MM)
|
Clinical conditions
|
+
|
+
|
+
|
Healthcare provider using standard MDR form
|
ICD-MM
|
κ = 0.219
|
Mgwadere et al., 2016 (1)
|
Malawi
|
86
|
Facility-based MDR
|
No
MDR
International Classification of Disease Maternal Mortality (ICD-MM)
|
Clinical conditions
|
+
|
+
|
+
|
Healthcare provider using standard MDR form
|
ICD-MM
|
κ = 0.37
|
Mgwadere et al., 2016 (2)
|
Malawi
|
151
|
Facility-based MDR
|
No
Expert panel using ICD-MM
|
Clinical conditions
|
+
|
+
|
+
|
INTERVA-4
(electronic instrument VA)
|
ICD-MM
|
κ = 0.66
|
There were two studies which applied verbal autopsy (VA) using electronic instruments, known as INTERVA-M and INTERVA-4 (20, 25). In one study, the VA electronic instrument of INTERVA-4 was assessed as a comparator instrument to assign cause of maternal death using ICD-MM for healthcare providers in facility-based MDR. The agreement between the expert panel using ICD-MM and probabilistic model of INTERVA-4 was substantial (κ = 0.66, 151 maternal deaths). However, INTERVA-4 did not identify the contributing factors of maternal mortality (25).
The computer-probabilistic model (computer-coded VA) adapted from the general InterVA model to interprete VA data for deaths of women of reproductive age provided a consistent and reliable interpretation. The consistency of the InterVA-M model had a substantial kappa score (κ = 0.8, 258 maternal deaths) to assign a specific cause of maternal mortality (20).
Clearly established standards of the WHO coding of maternal deaths are required to classify the causes of maternal deaths. The articles on MDR conducted before 2010 used the physician review diagnosis as a main standard. This systematic review, in the context of M-DET, showed that the articles on MDR conducted after 2010 and before 2012 explicitly mentioned the use of ICD 10. Similarly, the study on MDR conducted in and after 2012 were shown to use the ICD-MM. None of the articles mentioned the utility of clinical standards to identify gaps and highlight deficiencies in the MDR process.
Table 2. Meta-analysis of maternal death review instruments
Meta-analysis
|
Pooled kappa
|
Random effect model
|
Heterogeneity test
|
95% CI
|
p value
|
I²
|
p- value
|
Lower
|
Upper
|
|
MDR instruments
|
0.72
|
0.43
|
0.99
|
<0.001
|
96.19%
|
<0.001
|
Table 2 represents a strong pooled of MDR instruments 0.72 (95% CI 0.43–0.99) in conducting this review. However, the agreement among the studies was considerably homogenous I2 = 96.19% (p < 0.001). The graphical funnel and forest plots (Figs. 2 and 3) were asymmetrical indicating a possible publication bias.
The subgroup analysis based on the type of maternal audit (Table 3) highlighted the strong pooled kappa of VA 0.89 (95%CI: 0.52–1.25). Although VA and facility based-MDR had a correlation to assign cause of maternal death, there was a significant heterogeneity present in VA, I2 = 96.96% (p < 0.001) and facility based-MDR, I2 = 86.58% (p < 0.001).
Table 3
Sub-group analysis based on type of maternal audit
Type of maternal audit
|
Weight ratio
|
Pooled Kappa
(95% CI)
|
p-value
|
Heterogeneity test
|
p value
|
I2 (%)
|
Verbal autopsy
|
|
0.89 (0.52–1.25)
|
< 0.001
|
< 0.001
|
96.96
|
Fottrell et al., 2007
|
1.10
|
|
|
|
|
Midhet et al., 2008
|
0.40
|
|
|
|
|
Montgomery et al., 2011
|
1.26
|
|
|
|
|
Montgomery et al., 2012
|
0.79
|
|
|
|
|
Facility-based MDR
|
|
0.48 (0.14–0.82)
|
0.005
|
< 0.001
|
86.58
|
Owolabi et al., 2014
|
0.22
|
|
|
|
|
Mgwadere et al., 2016(1)
|
0.39
|
|
|
|
|
Mgwadere et al., 2016(2)
|
0.79
|
|
|
|
|
The potential sources of heterogeneity (subgroups of regions, sample-size, and year of publications) were tested by meta-regression methods. There was significant association found among these variables (I2 > 92%, p value for heterogeneity < 0.001) (Table 4).
Table 4
Meta-regression analysis results
Type of maternal audit
|
Pooled Kappa
(95% CI)
|
p-value
|
Heterogeneity test
|
p-value
|
I2 (%)
|
Region
|
|
|
|
|
Asia
|
0.82 (0.34–1.29)
|
< 0.001
|
< 0.001
|
96.92
|
Africa
|
0.64 (0.25–1.03)
|
< 0.001
|
< 0.001
|
94.56
|
Sample size
|
|
|
|
|
< 100
|
0.40 (0.11–0.45)
|
< 0.001
|
< 0.001
|
98.99
|
≥ 100
|
0.87 (0.59–1.15)
|
< 0.001
|
< 0.001
|
95.79
|
Year of publication
|
|
|
|
|
< 2011
|
0.75 (0.51–1.21)
|
< 0.001
|
< 0.001
|
94.54
|
2011–2012
|
1.00 (0.57–1.25)
|
< 0.001
|
< 0.001
|
95.41
|
> 2012
|
0.58 (0.07–1.08)
|
0.025
|
< 0.001
|
95.24
|