Study design
In order to evidence the effects of noise and task interruptions on nurse triage accuracy, we will carry out a 2x2 factorial randomised controlled trial (78). The use of a factorial design is justified by the fact that two independent variables will be manipulated (noise and task interruptions) and that we wish to examine not only the effect of each one on the dependent variable (nurse triage accuracy), but also their combined effect (78-80). The factorial design will follow the CONSORT guidelines (81) and will lay out the structure of the trial and the choice of control, an analysis of study benefits and harms, the quality and reliability of the intervention, a description of the population, the randomisation procedure, and the statistical analysis plan.
This factorial design will allow us to create four groups: one control group and three experimental groups. While triaging the clinical vignettes, nurses in the control group will be exposed to no distractors, those in experimental group A will be exposed to a noise distractor, those in experimental group B will be exposed to a task interruption distractor, and those in experimental group C will be exposed to both types of distractor . The nurses will be block-randomised across the four groups by a computer program. The groups will be of equal size or as similar as possible in this regard.
The study design comprises longitudinal measures. We will collect sociodemographic and personal data from the nurse participants before they begin evaluating the clinical vignettes. Then, during the evaluation of each clinical vignette, the following data will be systematically gathered: 1) emergency level assigned; 2) level of perceived confidence in emergency level assignment; and 3) duration of each clinical vignette evaluation. Upon completing the evaluation of the 20 clinical vignettes, the nurse participants will be asked to complete a questionnaire on the acceptability of the SG (Table 1
Table 1 Data collection for ED nurses
|
|
|
|
|
|
|
|
|
|
|
|
TIME POINT
|
|
Enrollement
|
Allocation
|
Intervention (SG)
|
Close out
|
|
|
|
|
|
|
20 clinical vignettes (CV) in 2 hours
1er CV 2 emeCV 3emeCV 4emeCV 5emeCV 6emeCV…..
|
|
|
ENROLMENT
|
Project presentation
|
x
|
|
|
|
|
|
|
|
|
|
|
|
to Swiss Triage Group
|
|
|
|
|
|
|
|
|
|
|
|
|
ED units recruited
|
x
|
|
|
|
|
|
|
|
|
|
|
|
Informed consent obtained for
|
x
|
|
|
|
|
|
|
|
|
|
|
|
ED Nurses
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ALLOCATION
|
Randomization of nurses
|
|
x
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
INTERVENTION
|
Control group (CG)
|
|
|
|
x
|
x
|
x
|
x
|
x
|
x
|
|
|
|
Experimental groups ABC (EG)
|
|
|
|
x
|
x
|
x
|
x
|
x
|
x
|
|
|
ASSESSMENT
|
|
|
|
|
|
|
|
|
|
|
|
|
ED NURSE (CG and EG)
|
QUANTITATIVE DATA
|
|
|
|
|
|
|
|
|
|
|
|
|
Sociodemographic questionnaire
|
|
x
|
|
|
|
|
|
|
|
|
|
Level of urgency
|
|
|
|
x
|
x
|
x
|
x
|
x
|
x
|
|
|
|
Chief complaint SETS
|
|
|
|
x
|
x
|
x
|
x
|
x
|
x
|
|
|
|
Level of confidence
|
|
|
|
x
|
x
|
x
|
x
|
x
|
x
|
|
|
SERIOUS GAME
|
Visual analogic scale
|
|
|
|
|
|
|
|
|
|
x
|
|
|
Attradkiff scale
|
|
|
|
|
|
|
|
|
|
x
|
|
Population and sampling
This multi-site study will be carried out in ED where the SETS® is used. This is the case in 20 private and public care facilities in the five cantons of French Switzerland (Geneva, Vaud, Fribourg, Jura, Neuchâtel). The pilot study population will consist of all nurses who perform the function of triage nurse in these facilities. As at this day, this corresponds to an accessible population of 454 nurses. The eligibility criteria will be based on those used in previous studies of triage accuracy (25, 34, 83). More specifically, to participate in our study, nurses must: 1) perform triage in one of the ED where the SETS® is used and 2) consent to participate. There will be no exclusion criteria in order to obtain a broad panel of nurses. All that matters for the study is that a nurse performs ED triage. The list of nurses will be drawn up by the head nurses in each of the participating units and this list will be block-randomised across the four groups. Assuming an accurate triage rate of 0.85 for the control group, a decline of 0.1 in the experimental groups, and an intraclass correlation (that is, between vignettes triaged by a same nurse) of no more than 0.03, we estimated through simulations that each group would need to comprise at least 20 participants to obtain a power of at least 0.80 in order to answer the primary research question. Consequently, we will aim to form a convenience sample of at least 80 nurses and we will cease recruitment once this target is reached.
Procedure
The study will follow a four-step procedure. First, the SG will be constructed and the clinical vignettes developed in conjunction with the various experts involved in the study. Pre-tests have been planned in order to regularly assess the game’s scalability, the relevance of the clinical vignettes selected, and the continuous analysis of the time required to evaluate the 20 vignettes. Once the vignettes thus created are stabilised, the clinical experts will, in concert with the research team, assign to each a gold standard. Second, all of the care facilities with an emergency unit in the five cantons of French Switzerland (Geneva, Vaud, Fribourg, Jura, Neuchâtel) will be contacted to validate their interest in participating in the study. To date, five facilities have confirmed their intention to participate, which represent an accessible population of 65 nurses. Third, each nurse who consents to participate in the study will receive an information and consent form. The definitive list of participants will be drawn up after consent forms are signed and each nurse participant will then be assigned an identification number. These numbers will be used to randomise the nurses across the four groups. Fourth, the study data will be collected. To this end, the head nurses of each participating unit will, together with the research team, establish a plan to deliver the SG in their unit over a period of three months. SG delivery will follow the procedure laid out below in the section titled “SG: construction and delivery”. After their two-hour SG session, the nurse participants will return to their workplace.
SG: construction and delivery
The SG comprises three components: graphic interface, clinical vignettes, and distractors.
- Graphic interface: The purpose of the SG is to simulate the conditions of an ED triage station as authentically as possible. The “ED triage” SG will be developed on and operated from an open-source platform called Wegas (http://www.albasim.ch). To develop the SG and allow it to evolve, the research team will use a logical graphic interface that may include audio-visual elements adaptable to needs and scenarios. This graphic interface will consist of a virtual waiting room capable of containing different animated sequences, such as the arrival of patients and care staff. A triage station will be recreated, equipped with all the devises used by nurses under the circumstances (e.g. triage form, clock).
- Clinical vignettes: A series of 20 interactive clinical vignettes will be developed by the research team based on real cases, revised by a group of four experts (two staff physicians and two nursing experts), and incorporated in the SG. The clinical vignettes will be constructed following the three quality guidelines proposed by Evans et al. (84): 1) each vignette must simulate situations faced by participants, which is the case in this study; 2) each vignette must be different and entail a specific decision to be made, which in our case is to assign an emergency level to each vignette; and 3) the results obtained from using the vignette must be transferable to real-life triage situations, which makes for a better generalisation of the results. For our study, all the clinical vignettes will involve the medico-surgical issues most commonly encountered in ED and will be generalisable to other French-speaking emergency units. Finally, for each clinical vignette, the emergency level will be validated by expert consensus in strict compliance with the criteria and definitions of the SETS® (90). This will constitute the gold standard.
- Distractors: Noise and task interruptions will be selected in connection with the scientific literature, using, for example, the instrument developed by Johnson and colleagues (2014) for classifying the different task interruptions into categories (38). Interruptions will be introduced in the 20 clinical vignettes for the experimental groups. The distribution of task interruptions (type, number and duration) will follow a predetermined sequence and will therefore be perfectly reproducible from one vignette session to another. The SG will allow nurse participants to choose to respond or not to some task interruptions (e.g. an incoming telephone call) but will require them to respond to others (e.g. a patient inquiring about the wait time). The noise will correspond to the soundscape (observed value) of triage stations. Given that no previous study has ever proposed a categorisation, audio recordings will be made at several stations and the different types of noise obtained will be analysed. The research team will modulate the noise exposure condition by varying the form, length of exposure, and intensity of the ambient noise (e.g. conversation, telephone ringtone). The intensity of the noise exposure will range from 35 dB (A) to 85 dB (A), the maximum level at which no auditory protection is required (85). To create a sound immersion and eliminate extraneous noise, nurses will be required to wear headphones during the SG session.
SG delivery will comprise several stages. First, the nurse participants will receive a 15-minute information session led by a member of the research team to familiarise themselves with the SG. During this session, the nurse participants will be able to ask all the questions they want to familiarise themselves as best possible. Second, the nurse participants will have the chance to test the SG on two clinical vignettes that will not be included in the analyses. During this practice run where the nurses will be equipped with headphones, the members of the research team will be on hand to answer questions, if any, and to provide help with how to use the SG, if needed. Once the practice run is completed, each nurse will be able to begin evaluating the vignettes when ready. The 20 vignettes will be presented in a pre-established order. The nurses will have two hours within which to complete the 20 evaluations, which corresponds to the average number of patients triaged at an emergency department over this lapse of time. The SG will be delivered in the computer room of each participating site following a schedule established by the head nurses in each participating unit. In all, at each game session, a maximum number of four nurses will be able to participate, based on the number of computers available at each site. All the data collected during the actual game phase will be automatically recorded and saved by the SG in a swiss located computer server. Members of the research team will be on site to provide technical support, if needed, and to document any anomaly that might occur during the SG. Finally, once the allotted two hours have elapsed, each nurse participant will stop the game session. If all 20 vignettes have not been evaluated by then, the nurse participant will complete the vignette presently being evaluated before stopping the game. All of the actions taken by the nurse participants will be timed systematically by the SG and saved to log files for later analysis.
Instruments of measurement: Outcome measures
Sociodemographic data, both personal (gender, age, family situation) and professional (employment status, total number of years of experience, number of years in current department), will be collected through a questionnaire developed on the basis of elements gathered in the course of previous studies on the subject (29, 34, 53).
The clinical decision making of the nurse participants will be judged on the emergency level that they assign based on the SETS® criteria. This scale was initially developed in Geneva and called the Geneva Triage Scale. Presently, the SETS® has four levels: acute = 1, urgent = 2, semi-urgent = 3, non-urgent = 4. Following their clinical reasoning, nurses assign patients an emergency level from 1 to 4. The scale has been the focus of various independent studies (25, 87, 88) where computerized clinical vignettes were used with ED nurses and paramedics. It is currently recommended by the Swiss Society of Emergency and Rescue Medicine (24) for ED patient triage and widely used in Switzerland, France and Belgium.
Nurse level of confidence in their clinical decision making will be measured using a visual analogue scale from 0 to 100 (89). This scale will be used by nurses after each emergency level assignment immediately after validating it. The question asked will be: “Now that you have completed this clinical vignette, in your opinion, what is your degree of confidence in the emergency level that you have assigned?” Nurses will rate their confidence from 0 to 100, with 0 corresponding to “I have no confidence at all in my decision” and 100 to “I have full confidence in my decision”. Visual analogue scales allow measuring the intensity of a subjective experience and are widely used in clinical settings (79). The researchers in a study where this scale was used by 69 nurses in a triage situation reported no problems with its utilisation (53).
The feasibility of the SG will be assessed on the basis of criteria drawn from Sidani and Braden (76) and Feeley et al. (58), including accessibility of target population, appropriateness of inclusion and exclusion criteria, nurse participation rate, nurse withdrawal rate after starting SG, presence and frequency of problems during the SG (understanding, utilisation, clarity), presence and frequency of missing data and outliers, and nurse participant satisfaction with the SG.
The acceptability of the SG will be measured using a French version of the self-administered Attrakdiff 2 (90) developed initially by Hassenzal and colleagues in German (91). This 28-item scale allows evaluating the hedonic and pragmatic qualities of interactive systems such as SGs. Each item takes the form of a seven-point scale (-3 to +3) on which to rate a quality according to semantic differentials, that is, a pair of antonyms. It comprises four subscales: usability, functionality, social impact, and attractiveness. For each item, the respondent must choose between seven answers bookended by the semantic differentials. A mean score and standard deviation are calculated for each dimension taking account of a recoding of certain inverted items (90). The values -1, 0 and 1 are considered neutral. Dimensions are deemed positive if scored +2 or +3 and negative if scored -2 or ‑3, in which case the SG needs to be improved. The psychometric properties (validity and reliability) of the French-language scale are entirely satisfactory, having obtained a Cronbach’s α of 0.75 for each of the dimensions (90). A supplementary question in the form of a visual analogue scale from 0 to 100 will be added to examine the nurse participants’ perception of the SG’s realism relative to their professional activity.
Data analysis plan
The analysis units will be the nurse participants (expected N = 80) for the descriptive analyses and the triage scale scores assigned (number of nurses x number of vignettes: expected N: 80 x 20 = 1600) for the correlational analyses and some descriptive analyses. To answer the research questions, the following data analysis plan is proposed. First, the collected data will be verified (compliance with inclusion criteria, identification of missing data and outliers). Second, the data on the nurses (sociodemographic and professional) will be analysed via descriptive statistics, both univariate (mean, median, standard deviation, interquartile range, and absolute and relative frequency) and bivariate (contingency table and marginal frequency). Third, triage accuracy will be measured by the level of agreement obtained between the answers given by the nurses and the gold standard established by the experts. For each nurse, the scores assigned to each clinical vignette will be compared against the gold standard. The result will be a three-category multinomial variable: accurate triage (nurse score same as gold standard), overtriage (score higher than gold standard), and undertriage (score lower than gold standard). Over- and undertriage frequencies will be used to describe the triage accuracy of the four groups, that is, the control group and three experimental groups: Noise (A), Task interruption (B), and Noise and task interruption (C). Fourth, to examine the individual and combined effects of the distractors on the triage accuracy of the nurse participants, the groups will be compared to one another using a random-intercept multinomial regression model. For all the analyses, the statistical significance level will be p ≤ 0.05. All the data will be analysed using the R statistical software (93).
Ethical considerations
Each nurse from the emergency units selected for the study will receive a written information letter explaining how the study will be conducted, what their participation entails, and what data protection measures will be taken. Only team research will have acces to anonymous data. All data will be canceled after analysis data. Each nurse will then take the time they need to decide whether to participate in the study, without the decision having any consequence whatsoever for their career. To participate in the study, the nurses will have to sign a consent form, which will be stored in accordance with the recommendations of the Swiss Human Research Ethics Board (Canton of Vaud, Switzerland). The time that the nurses spend evaluating the clinical vignettes with the SG will count as work hours.