Analysis of the data collected revealed important structural differences between spontaneous medical sign-out and the gold standard, and between spontaneous medical sign-out and the I-PASS tool.
First, we noted that the relevance and completeness rates are very strongly correlated (R2 = 0.91) and are on average relatively low (respectively 37.2% and 51.9%). Thus, more than 60% of the information transmitted by the participating physicians was not considered necessary by the group of experts, and almost half of the information determined as essential was missing.
This result can be explained in part by the fact that these two notions, relevance and completeness, are calculated in relation to a consensus among experts on the ideal content of each sign-out. However, this consensus depends on the reflective path of these experts, which can be multiple. According to a study by B. Charlin (15), the results of a group consensus differ by 59% from the answers that these same experts would have had individually. Faced with the same clinical situation, the ways of thinking may vary from one clinician to another (16),(17)and diagnostic accuracy is influenced more by thinking about diagnostic hypotheses than by a precise list of information to be sought (18). Thus, much information transmitted by some participants, but absent in the gold standard, seemed to be quite relevant even if it was not considered essential by the gold standard. The "irrelevance" group might therefore be a mixture of useful information for management, and of useless or even erroneous information. Therefore, in a secondary analysis, we created a second, expanded gold standard that included all potentially useful information for the management of the referred patient, justifying this classification with data from the scientific literature. The extended relevance rates of the participants recalculated in this way averaged more than 90%, which means that the amount of useless or erroneous information is relatively low.
The low completeness rate might be explained by an overestimation of the physician regarding the effectiveness of her sign-out. Indeed, one study (19) showed that the most important element of a clinical case is not transmitted in 60% of cases, although the transmitting physician is convinced that it has been correctly understood. Studies in communication psychology (20),(21) confirm that the speaker systematically overestimates what she thinks the listeners have understood. Since the situation is clear to her and she knows what she wants to transmit, she tends to think that her ideas will be easily perceived by her interlocutor, thus remaining unaware of the omissions made, which can be a source of communication errors (22).
In our analysis, the completeness rate was not higher in the supervisor group. This may suggest that, with increasing experience level, information is more easily implied (23). Indeed, even though each participant was instructed to put herself in the role of a resident passing on information to another resident, it is likely that this was less natural for supervisors, especially the more senior doctors. The study by V. Chang mentioned earlier shows that professional experience does not improve the effectiveness of sign-outs, and that the rate of misinterpretation of the other person's understanding remains very high (19). There is currently no teaching about medical communication in the residency, so it is not surprising that it does not seem to improve with professional experience.
The positive influence of the duration of sign-out on their completeness was expected, since the longer the sign-out, the more likely it is to contain all the essential elements. However, it should be remembered that the main goal of sign-out is efficiency (number of features signed out in relation to duration) and that lengthening their duration is therefore not the solution to the problem. The challenge for optimal hand-offs lies in the process of selecting the relevant information for a given clinical situation while remaining concise. An trade-off must be made to maintain a balance between too much detail (24) and shortcuts or omissions that can lead to misunderstandings. Working on the structure of hand-off and sign-out’s communication with a mnemonic tool to avoid omissions and facilitate comprehension seems promising for improving their quality (10). In particular, the last category of I-PASS (2nd S) could prove an effective tool to address this source of error. Indeed, a synthesis by the receiver offers the possibility of correcting potential misunderstandings or clarifying certain elements.
Furthermore, we were not surprised to observe a very uneven distribution of the different I-PASS categories both in the gold standard and among all participants. Indeed, category P includes many elements (patient's personal data, clinical history, reason for hospitalization, potential diagnoses, current clinical situation, stages of management) that are necessary for efficient sign-out. The other three categories are equally important but include fewer elements. This categorization specific to the I-PASS mnemonic tool is precisely intended to emphasize the importance of communicating about patient's severity state (I), next steps in management (A) and/or anticipation of potential complications (S), which are the elements most often omitted during hand-off between caregivers according to previous studies (11).
However, if we compare the distribution of the I-PASS categories among the 8 clinical cases within the gold standard, we can see that the S category is missing in three clinical cases (4, 7 and 8), which means that, in these situations, no recommendation was clearly stated. For case 4 (onset of retrosternal pain in a patient with heart failure), this was because the gold standard suggested going directly to the patient before suggesting any anticipation. On the contrary, some participants advised calling the cardiology department based on the patient's clinical state and current electrocardiogram (8/20), and others recommended advancing the planned coronary angiography based on specialist advice (6/20). For case 7 (fever in a hyponatremic patient), both the gold standard and most participants (18/21) did not share a recommendation for a possible complication, considering that the patient was stable and that the necessary complementary examinations had already been ordered. The gold standard did not suggest any anticipations for case 8 (medication error), judging that close monitoring would be sufficient for this patient. However, more than one third of the participants reported that they would consider a transfer to the intermediate care unit if the patient's hemodynamic status worsened or if they were advised to do so by a supervisor (8/19). Category I was also absent from several clinical cases in the GS (cases 2, 3, 7 and 8), for which no notion of the severity of the clinical case was transmitted.
Given that these two elements have proven their usefulness in improving communication between caregivers (25),(26),(27) and are probably not yet part of the habits of the HUG internal medicine department, it could be advantageous to introduce them explicitly into the training of the internists.
The comparison between the gold standard and the sign-outs of the 30 participants revealed a significant difference in category A, which was underrepresented among the latter. Indeed, the participants gave less relevant information about the next steps in management than was ideally expected. For example, in case 3, none of them thought to stop ibuprofen when a skin rash with a strong suspicion of allergic etiology appeared. In case 8 (patient receiving a beta-blocker and an anti-diabetic medication by mistake), only one participant thought of suspending the next dose of the patient’s usual beta-blocker.
Conversely, the proportion of items in category I was significantly greater among participants. One reason for this difference may be that the focus group defined the gold standard items in a shared discussion and did not follow the full on-call simulation scenario: for example, the notion of priority among different patients that is part of category I was not explicitly present in the gold standard, whereas in the actual simulation, the investigator asked the participant whether there was a patient to be seen first.
Analysis of the I-PASS categories’ sequence revealed that the general pattern of spontaneous sign-out followed the order P-A-S-I if all categories were represented, with many variations in the second part of the sign-out (A-S-I) during which P category codes regularly intersected the other categories.
About half of the communication ended with category I because participants specified the order of priority between different patients at the end of the sign-out. However, the I-PASS tool suggests starting the sign-out with this category to directly give the degree of severity to the interlocutor, who will listen differently according to this information and directly identify patients at risk of complicating their condition in the next few hours (26). In our study, only one participant (P25) systematically followed this approach.
Our results also showed that the variability in the sequence of the I-PASS categories depended more on the clinical case than on the participant, especially for the S category that contains the participant's reflection to prevent a future problem. For example, for clinical case 7 (fever in a hyponatremic patient), 84% of the participants did not suggest anticipating potential complications. This can be explained on one hand by the fact that they had already taken the necessary actions (fever monitoring) and on the other by the complexity of managing hyponatremia. For clinical cases 2, 3, and 6, category S information was only present in some of the participants who had received a nurse call for this patient. Thus, in the absence of complications (nurse call), none anticipated potential complications for these patients. Conversely, anticipations were frequent in case 5, notably because some were already present in the initial hand-off (CPAP in case of desaturation, NIV if insufficient) and were therefore only repeated by the participants. Similarly, for case 1, the results of the paracentesis were to be tracked and, depending on these, the participant proposed to her colleague to start an antibiotic treatment (S). For this same case, participants also often shared their suggested response to complications (S), believing that this patient was fragile and at risk of deteriorating in the following hours. Thus, the perception of instability would prompt participants to address this category’s element.
Moreover, more codes from categories A, S, and I are present in the sign-outs with nurse call, which illustrated an acute complication that had needed to be taken care of. Hence, for a clinical case without an acute complication during their shift, many participants spontaneously omitted information about the urgency state, the to-do list and anticipation. The data has however shown that the omission of this information could be a source of error and is thus important in the care transitions.
As for the 2nd S of the I-PASS mnemonic tool (receiver synthesis), the study was not designed to specifically study it since the receiver was part of the investigation team. However, based on the initial hand-off given to the participant in a standardized way, we realized that only one participant rephrased the received hand-off to ensure proper understanding. This synthesis is therefore not yet part of the local habits and should be actively promoted in training.