To the best of our knowledge, this is the first study to explicitly examine the performance of an OTS as applied by midwives, as well as by RNs without experience in obstetrics. Our findings suggest that the GOTS has good IRR and is a reliable tool for triaging obstetric patients seeking emergency care, regardless of whether assessors have scarce or extensive experience of triaging and managing obstetric patients. It is thus applicable for triaging obstetric patients presenting for emergency care at obstetric units and at general EDs.
The purpose of triage is to identify patients with time-sensitive, severe conditions and distinguish them from less urgent patients. The system facilitating these assessments should be as reliable as possible, enabling reproducibility of assessments made under similar conditions.(18) In this study, there were slight differences in IRR between midwives and RNs. This is to be expected, since the clinical experience of the triage midwife/RN will inevitably play a role in assessment of the patient. Indeed, previous research has shown that the triage process is dependent on both external factors, such as work environment, but also on internal factors, such as individual capacity.(19–21) Triage systems facilitate triage but do not constitute the entire triage process. Reliable triage systems, such as the GOTS, are even more crucial in order to avoid variation in assessment, which in turn decreases the risk of unequal care and of overlooking severely ill patients.
Of the 418 assessments, 82 (21.1%) and 36 (9.3%) were under- and overtriaged, respectively. This highlights that IRR is important in evaluating the system, but it does not necessarily correspond to the clinical significance. The clinical relevance of coherence in assessments must be taken into consideration when interpreting the numerical Kappa value. Failure to reach consensus concerning triage level will result in over- and undertriage, affecting both patient safety and resource allocation. In addition to establishing a good IRR, we thus also analysed the frequency of and the reasons for over- and undertriage. Clinically, the most immediate and severe triage failure is related to undertriaging patients while crossing the unstable/stable barrier i.e., triaging a red or orange patient as yellow or lower in the GOTS. In our study, this type of undertriage was seen in 15.9% (27/170) of the cases that were assessed as red or orange by the consensus group. When analysing these cases, it was evident that undertriage was predominately caused by two key factors - limitation of the paper case study design and failure to apply the system correctly. Several of the undertriaged assessments occurred in cases with bleeding and/or abdominal pain. These cases were also overtriaged by some participants, indicating that these symptoms may be difficult to assess on paper. These difficulties were addressed spontaneously by the participants after completing all assessments. They pointed out that they would have been able to assess the amount of bleeding as well as the level of pain in a clinical setting.
The other main reason for misclassification was not acknowledging abnormal vital sign parameters. This highlights two aspects. Firstly, there is a need for continuous education about the GOTS. Specifically, if a vital sign parameter indicates a higher acuity level, it is this level that must be chosen. Secondly, vital sign parameters may be underestimated and the interpretation of their significance may vary depending on the assessor’s background. Previous research has shown that assessing vital sign parameters in the non-obstetric population strengthens the ability to adequately triage patients.(22, 23) However, in obstetrics, it is mainly blood pressure and temperature that have traditionally been taken into account when assessing patients. In this study, although sample size was small, a difference was seen between the midwives and RNs. Midwives tended not to acknowledge the significance of an elevated heart rate while RNs tended not to react to blood pressure exceeding 160/110 mmHg, a blood pressure level that is considered somewhat elevated in general emergency medicine but of the utmost significance in obstetrics. Addressing all vital sign parameters in obstetric emergency care may challenge assessments previously perceived as correct. This thus generates a potential problem in trusting the system and applying it as intended. Further research is needed to establish whether all vital sign parameters are essential in triaging obstetric patients. As the physiological changes occurring during pregnancy enable obstetric patients to maintain normal vital sign parameters despite being critically ill (24), there is all the more reason to react when vital signs actually deteriorate.
Strengths and limitations
Previous research has highlighted the importance of triage nurses’ having experience in emergency care for proper implementation of the triage system itself.(21, 25) A strength of this study is assessing the IRR of the GOTS among participants with experience in emergency care but with varying experience in obstetrics. Moreover, assessment by a consensus group enabled an analysis of both over- and undertriage, in order to further assess the clinical significance of the IRR.
However, the study also has several limitations. As previously discussed, paper cases create difficulties in assessing symptoms such as amount of bleeding and level of pain, as confirmed by the participants. However, a previous study by Worster et al. established that there is moderate to high agreement in IRR between real-life and paper cases.(26) As the IRR in the consensus/real-life comparison was excellent, this might also have been the case in this study. On the other hand, IRR has been reported to be worse in paper case-based studies, compared to the same system tested in a live setting. (26)
The paper case design might be considered to be a strength of the study, allowing evaluation of all CCAs, as well as all triage levels. The study design also made it possible to test the assessment of patients triaged as red (immediate) and orange (urgent)– rare in the clinical setting but the most important to identify. Furthermore, the paper cases allowed the system’s IRR to be tested among both midwives and RNs, as well as coherent assessment of the patients.
Another limitation is the relatively small sample size, despite invitations being sent out to all staff members at the general ED and the obstetric ED. Triaging more cases would have increased the number of assessments. All study participation must be voluntary and it was hence not possible to include more assessors. It was not feasible to extend the recruitment period.
Like the absolute majority of triage systems, the GOTS was developed within the local context of guidelines and clinical setting, and the generalizability may thus be limited. The fact that the selection of cases did not correspond to the actual patient flow in the real-life setting may have biased the assessors to assess the cases according to the expected percentage of cases within each level of acuity. It was, however, deemed essential to evaluate all levels of acuity.