A Novel Algorithm-Driven Hybrid Simulation Learning Method to Improve Acquisition of Endotracheal Intubation Skills: A Randomized Controlled Study

doi:10.21203/rs.3.rs-731336/v1

Download PDF

Research Article

A Novel Algorithm-Driven Hybrid Simulation Learning Method to Improve Acquisition of Endotracheal Intubation Skills: A Randomized Controlled Study

https://doi.org/10.21203/rs.3.rs-731336/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 08 Feb, 2022

Read the published version in BMC Anesthesiology →

You are reading this latest preprint version

Background: Simulation-based training is a clinical skill learning method that can replicate real-life situations in an interactive manner. In our study, we compared a novel hybrid learning method with conventional simulation learning in the teaching of endotracheal intubation.

Methods: One hundred medical students and residents were randomly divided into two groups and were taught endotracheal intubation. The first group of subjects (control group) studied in the conventional way via lectures and classic simulation-based training sessions. The second group (experimental group) used the hybrid learning method where the teaching process consisted of distance learning and small group peer-to-peer simulation training sessions with remote supervision by the instructors. After the teaching process, endotracheal intubation (ETI) procedures were performed on real patients under the supervision of an anesthesiologist in an operating theater. Each step of the procedure was evaluated by a standardized assessment form (checklist) for both groups.

Results: Thirty-four subjects constituted the control group and 43 were in the experimental group. The hybrid group (88%) showed significantly better ETI performance in the operating theater compared with the control group (52%). Further, all hybrid group subjects (100%) followed the correct sequence of actions, while in the control group only 32% followed proper sequencing.

Conclusions: We conclude that our novel algorithm-driven hybrid simulation learning method improves acquisition of endotracheal intubation with a high degree of acceptability and satisfaction by the learners’ as compared with classic simulation-based training.

Anesthesiology & Pain Medicine

HybridLab

endotracheal intubation

learning outcomes

self-directed learning

peer-to-peer simulation

The science of education is constantly looking for novel ways to improve the learning process. In medical studies, teaching students’ practical skills that can be readily and safely applied in clinical practice is a major goal. Simulation-based training is an established, important learning tool in medical education facilitating the acquisition of necessary competences and has been reported to be more effective and superior to problem-oriented learning [1, 2]. However, there are some disadvantages. Acquiring robust skills that can be applied in clinical practice requires multiple repetitions of the skill in a simulated environment, which takes considerable time and teaching resources. It has been reported that a minimum of 50 practices are needed for successful endotracheal intubation skill acquisition [3].

Our learning lab (HybridLab® at the Lithuanian University of Health Science [LUHS]) has developed a learning method designed to use simulation resources more efficiently and to empower learners to exercise effective small group peer-to-peer simulation training sessions with direct or remote supervision of the instructors, either synchronously or after the video review (asynchronously). This method offers a well-structured and standardized learning pathway, which encompasses studies on an e-learning platform, peer-to-peer hands-on training sessions in the skill lab or simulation classes using carefully elaborated learning algorithms, direct feedback by peers, and assessment by a remotely working instructor [4–6]. Mobile technologies and algorithm-driven learning facilitates peer-to-peer learning, offers an opportunity to save time and human resources in the simulation centers, creates unique possibilities for learners to explore the benefits of autonomous and self-regulated learning, and develop new feedback and peer assessment techniques as well as leadership qualities. Interactive algorithms used in the learning process guide the novice learners in a step-by-step manner, helping them to create a well-structured mental pathway for decision making and/or execution of the procedure, obviating any possible learning mistakes.

The hybrid learning system is supported by two theoretical frameworks. Ericsson’s [7] Deliberate Practice model reinforces the concept that practice with feedback is essential to improvement. This feedback needs to include both self-reflective feedback as well as feedback from others (e. g. faculty, peers) as compared to a standard of performance. Bloom’s [8] Mastery Learning model includes breaking larger tasks into smaller elements and perfecting the performance of each element before moving on to the next element.

This hybrid training method was developed at the Crisis Research Center at LUHS. To our knowledge, this hybrid model is novel, with several published studies documenting its characteristics [5, 6]. There have been no similar studies published on endotracheal intubation skills.

Our hypothesis is that the hybrid learning technique is superior to conventional simulation-based learning techniques currently used in teaching airway management skills in the Anesthesia module to medical students at the postgraduate and/or resident training level. Therefore, the aim of our study was to compare our hybrid learning method with a classic simulation-based training for teaching endotracheal intubation skills.

Study population and design

The study was approved by the The Lithuanian University of Health Sciences Bioethics Commission no. BEC-MF-442. All procedures performed in studies involving human participants were in accordance with the ethics standards of the institutional and national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethics standards. All the participants gave written informed consent.

This was a randomized experimental design with 78 medical students in their 5th year course in the Anesthesia module and 22 first-year residents of Emergency Medicine and Anesthesiology. The study was carried out at LUHS from May 1st through June 1st, 2019. Randomization was performed prior to the teaching using a random number generator. Initially 50 students/residents were allocated to each study group (a) conventional simulation-based learning group (control group) and (b) the hybrid learning group (experimental group). Subjects who had previous similar practical skill training or experience were excluded from the study, as well as participants who failed to complete the course.

The participants in both groups studied the principles of safe airway management and practical skills of endotracheal (orotracheal) intubation. The learning aim and goals were identical for both groups, and the content of theoretical knowledge and skill set were aligned. Sixteen participants in the conventional simulation group and 7 participants in the hybrid group were excluded from the analysis. In total, 77 participants remained in the study (Fig. 1).

The conventional simulation-based training group teaching process consisted of lectures and practical skills with hands-on training sessions. The total duration of the course was six hours. During the simulation-based training sessions, the correct procedure for endotracheal intubation was demonstrated by the teacher on a manikin, which was followed by participants performing the intubation on the manikin under the teacher’s supervision.

The hybrid group initially studied the necessary theoretical material, lectures, videos, and algorithms on a virtual learning environment (which is comparable to Moodle and other widely used distance learning platforms). The students continued with the practical training only after passing an online test containing 10 questions with a perfect score. After the individual theoretical preparation, the subjects organized themselves into the groups of three for peer-to-peer practical skills training sessions in the HybridLab (Fig. 2). The pre-planned estimated duration of the skills training was six hours for each group. However, the HybridLab subjects were encouraged to adapt the learning time based on their needs and practice at their own pace as they had 24/7 access to the training lab and did not need to coordinate the timing with the technician and/or instructor. When learning practical skills, the subjects were using the same manikin and equipment as in the conventional simulation-based training group. However, this group had the supplementary support of hand-held tablets containing proprietary educational software, electronic scenarios and checklists for clinical situation assessment and feedback (Fig. 3) as well as the learning algorithms (Fig. 4), which enabled simulation and training in the absence of a technician or instructor.

Training sessions were video recorded on the cameras that were installed in the lab. The records of both the training and evaluation scenarios were viewed and evaluated remotely by the teachers. Standardized checklists were used for the evaluation, which were the same as what the subjects were using in training class. The evaluation and feedback of the teacher was provided by email.

Subjects were then asked to evaluate the course. Out of 77 participants, 58 (22 in conventional and 36 in hybrid group) completed the post-course survey. Two questions were asked regarding confidence and satisfaction: 1. After learning, I think I will pass the test successfully. 2. I enjoyed studying in this module. Each question had a 5-point Likert-type scale to evaluate the degree of student's agreement to a given statement, as follows: score 5 for strongly agree, 4 for agree, 3 for undecided, 2 for disagree, and 1 for strongly disagree.

For the final evaluation of both groups, endotracheal intubation procedures were performed on real patients in the operation room (OR) under the direct supervision of the clinical teacher (anesthesiologist). The only patients who were selected for the intubation were grade I and II patients according to the American Society of Anesthesiologists (ASA) [9]. To ensure the patient’s safety, each procedure step was closely monitored and if necessary, immediately corrected by an anesthesiologist. In addition, each step of the procedure was evaluated using the standardized assessment form (checklist). The checklist consisted of 16 steps (Fig. 3) and comprised the following parts: preparation for intubation, laryngoscopy, insertion of endotracheal tube, and verification of the placement of the endotracheal tube. The task was evaluated by the anesthesiologist who was blinded to the group the subject was enrolled in.

The sequence of endotracheal intubation procedure actions was assessed and compared between the groups. Also, participants were divided into groups according to the scope of activities performed ( ~ > 76%, 75 − 51%, 50 − 26%, < 25%). Additionally, the satisfaction and confidence responses were compared between the groups.

Statistical analysis

With an assumption that the difference in learning outcomes will be 30% better in the HybridLab group as compared with the conventional trained group, and with a statistical power of 0.8 and a risk of 0.05 for type-1 error, 30 participants were required in each group. For possible dropouts, 50 participants in each group were enrolled in the study.

To determine whether the data were normally distributed, a Shapiro–Wilk normality test was applied. As most of the data were not normally distributed, data were presented in medians (interquartile ranges) and rates. Also, a non-parametric independent-samples t-test for was used. The χ^² or Fisher’s exact test was used when comparing the proportions between the groups. P values < 0.05 were considered significant. SPSS software (SPSS, Chicago, Ill) was used for the calculations.

We included 77 participants in the final analysis: 34 in the conventional group (classical simulation-based training) and 43 in the hybrid group. Both conventional and hybrid groups did not differ significantly in respect to gender, age, the ratio of students or residents (Table 1).

Table 1

Comparison of participants’ data between the conventional and hybrid groups.
Gender n (%)	Conventional group (n = 34)	HybridLab group (n = 43)	p
Gender (female/male) (percentages)	23/11 (67/33%)	31/12 (72/28%)	0.803
Age (years)	22 (3)	22 (3)	0.526
Students/residents (percentages)	25/9 (74/26%)	30/13 (70/30%)	0.802

The results of the individual endotracheal intubation steps (16 actions) according to the checklist completed by the anesthesiologist in the OR with the subjects working with real patients are demonstrated in Table 2. Completion of all the actions of endotracheal intubation were significantly better performed by the subjects in experimental group when compared to those from the control group.

Table 2

Completion of 16 steps of endotracheal intubation comparison between conventional and hybrid groups. (ETT-endotracheal tube)
Action	Conventional group (n-34)	Hybrid group (n-43)	p
1. Checks equipment	56%	86%	0.003
2. Connects monitor	47%	91%	< 0.001
3. Preoxygenates correctly	27%	70%	< 0.001
4. Ensure correct patient position	77%	95%	0.019
5. Correct laryngoscopy	68%	95%	0.001
6. Describes laryngoscopy findings	71%	95%	0.003
7. Performs the right actions according to visual anatomical findings	62%	91%	0.002
8. Performs the right actions according to findings and situation	68%	95%	0.001
9. Correct insertion of the ETT (endotracheal tube)	47%	81%	0.002
10. Checks ETT depth	44%	93%	< 0.001
11. Inflates ETT cuff	65%	95%	0.001
12. Removes laryngoscope blade	56%	81%	0.015
13. Asks for CO₂ detector and checks it	47%	95%	< 0.001
14. Auscultates chest to assess position of ETT	50%	93%	< 0.001
15. Secures ETT	32%	86%	< 0.001
16. Assess ETT cuff pressure	9%	58%	< 0.001
Overall Median (interquartile range) Mean (standard deviation)	53% (23%) 52% (18%)	92% (13%) 88 (11%)	< 0.001

The correct sequence of actions in the OR was compared between the groups. The hybrid group was 100% accurate with all 43 subjects, while the conventional group had only 11 of 34 subjects follow the correct actions sequence (32%) (p < 0.001).

The distribution of correctly performed endotracheal intubation actions in both groups is presented in Table 3. In the hybrid group the number of correct actions performed between 76–100% was 91%, while in the conventional group it was 32%.

Table 3

Distribution according to correctly performed endotracheal intubation skill actions between the conventional and hybrid groups.
Actions performed correctly (approximate percentage)	Conventional group (n = 34)	Hybrid group (n = 43)	p
1 ⩾12 ( ~ > 76–100%)	11 (32%)	39 (91%)	< 0.001
2 11 − 8 (~ 51–71%)	9 (27%)	3 (7%)
3 7 − 4 (~ 26–50%)	10 (29%)	1 (2%)
4 ⩽ 3 ( ~ < 25%)	4 (12%)	0

A survey of student confidence and satisfaction showed that students’ confidence to pass the final test did not differ significantly between the groups. However, the satisfaction was significantly higher in the hybrid group as compared to conventional one (Table 4).

Table 4

Comparison of students’ confidence and satisfaction survey results between the conventional and hybrid groups.
Students’ survey		Conventional group (n = 22)	Hybrid group (n = 36)	p
Confidence question: 1. After learning, I think I will pass the test successfully.	5 (strongly agree)	4 (18%)	10 (28%)	0.259
	4 (agree)	6 (27%)	13 (36%)
	3 (undecided)	8 (36%)	7 (19%)
	2 (disagree)	4 (18%)	3 (8%)
	1 (strongly disagree)	0 (0%)	3 (5%)
Satisfaction question: 2. I enjoyed studying in this module.	5 (strongly agree)	3 (14%)	14 (39%)	0.02
	4 (agree)	5 (23%)	9 (25%)
	3 (undecided)	8 (36%)	6 (19%)
	2 (disagree)	2 (9%)	6 (17%)
	1 (strongly disagree)	4 (18%)	1 (0%)

The study demonstrated that a hybrid training method significantly improves the practical skill performance of endotracheal intubation as compared to the conventional simulation-based learning. We found that the overall average score of correctly performed endotracheal intubation procedure steps was on average 36% higher in the hybrid group than the conventional learner group. Similarly, other studies showed that simulation methods may improve training for acquiring practical skills [10–16]. This is in accordance with another study in which the hybrid training method was used. An overall average score was 96% of neonatal resuscitation skills among medical students [4] and in this study it was 88% of the endotracheal intubation skill. In the former study, the participants were evaluated by performing the skill on a manikin, while the current study allowed the subjects to apply the new skill on real patients. This suggests that hybrid training may allow achievement of similar learning results in both simulated and real clinical conditions.

When performing clinical skills such as endotracheal intubation, it can be said that there is little room for error; thus, perfect training requires an evaluation grade of 100%. Such high results were not achieved by either of our studied groups. One reason is that this was the first time that the participants had a chance to perform the intubation on real patients. However, the hybrid group performed perfectly on average 91% (varying between 76% and 100%) of the time, while in the conventional group it was only 32%. We believe that a better performance of the hybrid group could be attributed to the training in small groups of three (in comparison to 10–12 trainees per group in the classical training sessions) and the opportunity for each student to perform the skill many times at their preferred pace. The HybridLab setting allowed for repeated training sessions based on student preferences because they had 24/7 access to the skill lab and did not need to coordinate the time with an instructor or technician. In the classical simulation training model students and teachers alike are frequently restricted by time limits in face-to-face contact and the need to create equivalent learning opportunities for each student in the larger group due to differences in advance preparation and/or individuals’ capacity to acquire technical skills.

Peer-to-peer simulation sessions can be carried out at the pace needed by the learners and repeated until students develop automaticity and confidence; in the conventional class the instructor works with a larger group and clear time parameters, students are likely to have less hands-on time. Additionally, the better results might be due to the algorithm-driven and stepwise approach in HybdridLab training.

All participants in the HybridLab group followed the correct sequence of actions, while only 11 participants (32%) in the conventional group followed the correct sequence. The HybdridLab training being algorithm-driven and using a stepwise approach does not allow one to miss a single step during training. In this way, learning is based on 100% success, rather than making mistakes and reflecting on them after the procedure is completed. These findings are important, as patient safety necessitates a strict sequence of actions be adhered to in medical procedures [17, 18].

We found that subjects’ confidence about passing the final evaluation was not statistically different between the groups. However, satisfaction was significantly higher in the hybrid group. The hybrid learning has a well-prepared structure, good academic content and environment, and interactive practical training classes [4–6]. Overall, hybrid learning may increase the students’ satisfaction and motivation for learning and in this way improve the learning outcomes.

Limitations

There are several limitations in our study. Firstly, we were not able to evaluate the long-term retention of endotracheal intubation skills. However, this was done in another study that analyzed practical skill retention among medical students after 12 months and found that a mean percentage drop-off was 28% [4]. Second, a larger dropout of participants was observed in a conventional group as compared to the hybrid group. The reason for this could be that the participants in the hybrid group were more motivated to complete the course as has been suggested previously [4–6]. Additionally, the success of ETI was evaluated only once in OR and it might occur that in one of the groups there were more difficult patients. However, only ASA grade I and II patients were selected for the intubation in both groups.

The findings support the theoretical model of mastery learning integrated with deliberate practice. The hybrid model broke the highly complex task of endotracheal intubation into smaller steps that the hybrid learners progressed through, adding new steps one at a time. This was supported by peer-to-peer feedback that improved performance. With adequate time provided to achieve mastery at each step before moving on to the next step, the hybrid group was able to build a level of performance that exceeded the conventional group, who are generally time- limited and lack ongoing individualized feedback needed for continuous improvement.

We conclude that novel algorithm-driven hybrid learning method improves the acquisition of the endotracheal intubation skills as compared to conventional simulation-based training with a high degree of acceptability and satisfaction of the learners.

ETI endotracheal intubation

LUHS Lithuanian University of Health Science

OR operation room

ASA American Society of Anesthesiologists

Ethics approval and consent to participate

This study was approved by The Lithuanian University of Health Sciences Bioethics Commission no. BEC-MF-442. All procedures performed in studies involving human participants were in accordance with the ethics standards of the institutional and national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethics standards. All the participants gave written informed consent.

Consent for publication

Not applicable

Availability of data and materials

All data generated and analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

We declare that the author Aida Mankute works at the Crisis Research Center. Authors Dinas Vaitkaitis and Paulius Dobozinskas are the founders of the Crisis Research Center and HybridLab.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for profit-sectors.

Authors’ contributions

LJ, LS, PD, DV, DLR, AM conseived and designed the research; LJ, AM collected data and conducted the research;

AM, JS, MG, ZD analyzed, interpreted the data, wrote initial paper; All authors have read and approved manuscript.

Acknowledgements

Not applicable.

Morgan PJ, Cleave-Hogg D, Desousa S, Lam-McCulloch J. Applying theory to practice in undergraduate education using high fidelity simulation. Med Teach. 2006;28(1):e10–e15.
Steadman RH, Coates WC, Huang YM, Matevosian R, Larmon BR, McCullough L, Ariel D. Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med. 34;(1):151–157.
Buis ML, Maissan IM, Hoeks SE, Klimek M, Stolker RJ. Defining the learning curve for endotracheal intubation using direct laryngoscopy: A systematic review. Resuscitation. 2016;99:63–71.
Kudreviciene A, Nadisauskiene R, Tameliene R, Garcinskiene J, Nedzelskiene I, Tamelis A, Dobozinskas P, Vaitkaitis D. New HybridLab training method of clinical skills learning in neonatology: Students’ opinion. MedEdPublish. 2016;5(2):41.
Ignatavicius P, Gudaityte R, Dobozinskas P, Vaitkaitis D, Krikscionaitiene A, Dambrauskas Z. The novel platform for self-directed peer-to-peer medical simulation for development of the technical and non-technical competences. Ann Surg Educ. 2019;1:1001.
Kudreviciene A, Nadisauskiene RJ, Tameliene R, Tamelis A, Nedzelskiene I, Dobozinskas P, Vaitkaitis D. Initial neonatal resuscitation: Skill retention after the implementation of the novel 24/7 HybridLab® learning system. J Matern Fetal Neonatal Med. 2019;32(8):1230–1237.
Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(Supplement):S70–S81.
Bloom BS. Mastery learning. In: Block JH, editor. Mastery learning: Theory and practice. New York: Holt, Rinehart & Winston; pp. 47–63. 1971.
Abouleish AE, Leib ML, Cohen NH. ASA provides examples to each ASA physical status class. ASA Newsl. 2015;79:38–49.
Weller J, Robinson B, Larsen P, Caldwell C. Simulation-based training to improve acute care skills in medical undergraduates. N Z Med J. 2004;117(1204):U1119.
Noeller TP, Smith MD, Holmes L, Cappaert M, Gross AJ, Cole-Kelly K, Rosen KR. A theme-based hybrid simulation model to train and evaluate emergency medicine residents. Acad Emerg Med. 2008;15(11):1199–1206.
Okuda Y, Quinones J. The use of simulation in the education of emergency care providers for cardiac emergencies. Int J Emerg Med. 2008;1(2):73–77.
Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie WC. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: A case-control study. Chest. 2008;133(1):56–61.
Hallikainen J, Väisänen O, Randell T, Tarkkila P, Rosenberg PH, Niemi-Murola L… Teaching anaesthesia induction to medical students: Comparison between full-scale simulation and supervised teaching in the operating theatre. Eur J Anaesthesiol. 2009;26(2):101–104.
White S, Sykes A. Evaluation of a blended learning approach used in an anatomy and physiology module for pre-registration healthcare students. Proceedings of eLmL 2012 Fourth international conference on mobile, hybrid, and on-line learning. ThinkMind/IARIA; 2012;1–9.
Bouroche G, Bourgain JL. Preoxygenation and general anesthesia: A review. Minerva Anestesiol. 2015;81(8):910–920.
Routt E, Mansouri Y, de Moll EH, Bernstein DM, Bernardo SG, Levitt J. Teaching the simple suture to medical students for long-term retention of skill. JAMA Dermatol. 2015;151(7):761–765.
Sant’Ana GM, Cavalini W, Negrello B, Bonin EA, Dimbarre D, Claus C, Loureiro MP, Salvalaggio PR. Retention of laparoscopic skills in naive medical students who underwent short training. Surg Endosc. 2017;31(2):937–944.

Competing interest reported. We declare that the author Aida Mankute works at the Crisis Research Center. Authors Dinas Vaitkaitis and Paulius Dobozinskas are the founders of the Crisis Research Center and HybridLab.

Download PDF

Journal Publication

published 08 Feb, 2022

Read the published version in BMC Anesthesiology →

Editorial decision: Major revision
15 Nov, 2021
Reviews received at journal
01 Nov, 2021
Reviewers agreed at journal
27 Oct, 2021
Reviewers agreed at journal
07 Oct, 2021
Reviewers invited by journal
01 Oct, 2021
Editor assigned by journal
29 Sep, 2021
Editor invited by journal
29 Sep, 2021
Submission checks completed at journal
29 Sep, 2021
First submitted to journal
19 Jul, 2021

You are reading this latest preprint version

A Novel Algorithm-Driven Hybrid Simulation Learning Method to Improve Acquisition of Endotracheal Intubation Skills: A Randomized Controlled Study

Status:

Journal Publication

Version 1

Abstract

Figures

1. Background

2. Methods

Study population and design

Statistical analysis

3. Results

4. Discussion

Limitations

5. Conclusions

Abbreviations

Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1