A Randomized Controlled Trial of Simulation Training in Teaching Coronary Angiographic Views

doi:10.21203/rs.3.rs-1031601/v1

Introduction

Simulation technology has an established role in teaching technical skills to cardiology fellows, but their impact on teaching trainees to interpret coronary angiographic (CA) images has not been systematically studied. The aim of this randomized controlled study was to test whether structured simulation training in addition to traditional methods would improve CA image interpretation skills in a group of novices to advanced medical trainees.

Methods

We prospectively randomized 105 subjects comprising of medical students (N=20), residents (N=68) and fellows (N=17) from the University of Arizona. Subjects were randomized into a simulation training group which received simulation training in addition to didactic teaching (n=53) and a control training group which received didactic teaching alone (n=52). The change in pre and post-test score (delta score) was analyzed by a two-way ANOVA for education status and training arm.

Results

Subjects improved in their post-test scores with a mean change of 4.6 ± 4.0 points. Subjects in the simulation training arm had a higher delta score compared to control (5.4 ± 4.2 versus 3.8 ± 3.7, p=0.04), with greatest impact for residents (6.6 ± 4.0 versus 3.5 ± 3.4) with a p=0.02 for interaction of training arm and education status.

Conclusions

Simulation training complements traditional methods to improve CA interpretation skill, with greatest impact on novice trainees. This highlights the importance of incorporating high-fidelity simulation training early in cardiovascular fellowship curricula.

Educational Philosophy and Theory

Internal Medicine

simulation training

clinical competence

diagnostic angiography

Clinical educators are constantly innovating to address the challenge of effectively teaching students with ever increasing time constraints, medical complexity, and performance evaluation expectations¹. In recent years, technology-enhanced simulation training in health sciences has been established to have large effects on outcomes of knowledge, skills and behaviors with modest effects on patient related outcomes in comparison with traditional learning^2–4. The American Council for Graduate Medical Education (ACGME) statement of mandatory fellow participation in training has therefore recommended using simulation as a condition of accreditation for general cardiovascular and all cardiovascular sub-specialties. However, the exact methodology, implementation and educational objectives for simulation remains undefined^{5 6}.

In the cardiac catheterization laboratory, several small randomized and non-randomized trials have shown the superiority of global and technical performance scores encompassing multiple elements of performance relating to diagnostic coronary angiography^{7 8}, percutaneous coronary intervention^{9 10} and trans-septal puncture¹¹. Mentored simulation training may also have the potential to reduce procedural errors^{7 8 12}, contrast use, fluoroscopy and total procedure times^13–15. Small randomized controlled studies have also demonstrated their potential role in percutaneous coronary intervention training^{9 13 16}. There have, however, also been studies that have reported contradictory findings, primarily on clinical benefit, adding speculation to their overall advantages^{10 17 18}. and as a tool for assessing competency.

We therefore designed a randomized study examining, as a subject of research, the use of simulation by manipulating a virtual fluoroscopy C-arm in addition to a traditional method of learning via a lecture on coronary angiographic views on the achievement of a beginner competency in correct coronary angiographic view interpretation, focusing on early trainees. In contrast to prior studies, we decided to focus on a single, important, granular competency with which many early trainees traditionally struggle with and to utilize a function of endovascular simulation which has not been previously well studied – the representation of 3-dimensional virtual anatomy as a teaching tool in addition to standard fluoroscopic 2-dimensional simulated imaging. We hypothesized that the greatest impact of the protocol would be dependent on the level of training and therefore chose to study a broad range of learners, with a particular emphasis on novice or beginner trainees with either no or little prior exposure to the cardiac catheterization laboratory, represented by medical students and residents.

Study Population

One hundred and five trainees, comprising a spectrum of self-identified novice, beginner, intermediate and advanced trainees: medical students in their third and fourth year of training, internal medicine residents, family medicine residents, emergency medicine residents, and cardiovascular fellows from the University of Arizona met eligibility for the study and were consecutively enrolled from August 2016 to March 2017.

Ethics

The study protocol was in compliance with the Declaration of Helsinki and was approved by the hospital Institutional Review Board. All participants were briefed individually at the time of enrollment and they provided written informed consent.

Study Design

At study entry, all participants filled out an online survey to provide data on demographic and baseline self-reported visuo-spatial skills and were assigned a unique identification number at the start of the study. Next, they were administered a self-paced pre-intervention online test (pre-test), (see “Pre-test Coronary Angiographic Training Study”, Supplemental Digital Content 1). This test comprised of 30 de-identified still images and video clips of real angiographic films (that loop automatically every 5 seconds) in a multiple-choice format. Three questions tested for correct identification of each of the three coronary arteries with three multiple-choice question (MCQ) options. Twenty-seven questions tested for correct identification of the artery and projection with 9 MCQ options covering the six standard angiographic projections for the left coronary artery [left anterior oblique (LAO) caudal, LAO cranial, right anterior oblique (RAO) caudal, RAO cranial], and 3 for the right coronary artery (LAO cranial, AP cranial and RAO).

Study participants were then subjected to a stratified randomization process (Figure 1) based on their education status (medical student, resident, fellow) and divided into two training arms; a simulation arm which received simulation training in addition to didactic teaching (N=53) and a control arm which received didactic teaching alone (N=52). Blinding was not performed.

Simulation Interface

The Mentice VIST ® -C (Gothenburg, Sweden) is a portable, high-fidelity endovascular simulator, with good construct and concurrent validity, used in catheter training for coronary and peripheral interventions. This device is connected to a monitor and a laptop which runs the simulation software (VIST-8). The rest of the interface comprises of a dual foot switch for fluoroscopy and cine-angiography, and a syringe for simulated contrast injection (Figure 1). The simulator includes pre-designed coronary cases with angiographic data, and a single standard case with normal coronary anatomy was selected and used uniformly throughout the study. Additionally, the simulator has buttons and joysticks which enable the operator to virtually move the patient table and the C-arm to switch between angiographic projections akin to a real catheterization laboratory set-up. In addition to simulated standard fluoroscopic imaging, the user is able to switch to 3D visualized virtual anatomy.

Simulation Training

Simulation training consisted of a one-on-one mentored training on the simulator for 10-40 minutes (median: 20 minutes), duration of training was decided by the subjects themselves based on their learning pace. At the start of the exercise, they were instructed by a trained operator to manipulate the table and the C-arm, acquire standard angiographic projections and to recognize dynamic changes in the orientation of the coronary vessels in each of these projections, both in 2D and 3D modes. An instruction manual with conceptual checkpoints and the trained operator were available to the participants throughout the session if needed.

Didactic Teaching

Didactic teaching consisted of printed instruction material and a standardized ten-minute online video tutorial by Morton Kern, MD from his DVD “Cath Lab Essentials with Dr. Morton Kern” on identifying coronary angiographic projections¹⁹. Subjects could take notes during this session if they desired.

Evaluation of Post-Training Performance

At the end of the training session, trainees were subjected to the online post-test which comprised of the same 30 projections from the pretest, but randomly shuffled and the test was again self-paced. All pre- and post-test data obtained including test performance scores were archived into an online encrypted folder.

Statistical Analysis

Statistical analysis was performed using STATA version 14.2 (StataCorp., College Station, Texas). All data in this manuscript are represented as mean and standard deviation for parametric continuous variables and as proportions and percentages for categorical variables. Baseline characteristics were compared between control and simulation training groups with a t test for continuous variables and a Fisher exact test for categorical variables. ANOVA analysis was used to analyze the effects of training arm and education status and the interaction between these two factors. A two-sided p value≤0.05 was considered significant.

Study Population Characteristics

A total of 105 subjects were enrolled, comprising N=20 medical students, N=68 residents and N=17 fellows. Participants were predominantly male (59%) with a mean age of 30±4 years. Among the 68 residents, 55 were from Internal Medicine, 6 from Family medicine and 7 from Emergency medicine. A majority, 84%, of the participants self-reported playing a musical instrument, competitive sports, or video game routinely over the past year. In terms of handedness, the majority were right-handed (89%) with a small number of them reporting mixed handedness or being ambidextrous (8%). Prior to taking the study, 89% of the participants graded themselves as a novice or a beginner in coronary angiographic training of which 96% of them stated that they were not confident in accurately identifying coronary anatomy on an angiographic image. There were no significant differences in these baseline characteristics between the simulation and control arms (Table 1). After completion of the training experience, 50% rated the lecture as good or excellent. Among those randomized to simulation, 85% graded their satisfaction with the simulation experience as good or excellent and 98% agreed or strongly agreed that simulation training is essential in cardiovascular fellowship training. There were no adverse events.

Test Performance Scores

Out of a maximum score of 30, the pre-test score was 6.5 ± 2.5 for medical students, 6.5 ± 2.5 for residents, and 18.2 ± 4.1 for fellows (p<0.0001 for effect of education status). The change in score (delta score) from pre- to post-test is given in Table 2. Subjects improved in their post-test scores with a mean delta score of 4.6 ± 4.0, with subjects that underwent simulation training having a greater delta score (5.4 ± 4.2 vs 3.8 ± 3.7, p=0.04). The interaction of factors of training arm (simulation versus control) and education group (medical student, resident, fellow) were further evaluated in a two-way ANOVA. Education status alone was not significant (p=0.13). However, there was a significant interaction effect of training arm and education status (p=0.02), such that residents derived the greatest benefit from simulation training compared to control (6.6 ± 4.0 versus 3.5 ± 3.4, Table 2). Pre- and post-test scores by training arm and education status are shown in Figure 2.

The delta scores by angiographic view are shown in box plots in the Supplemental Figure 3 (see Supplemental Figure 3, Supplemental Digital Content 2). In this figure there is a visual impression that simulation best improved the identification of the RAO projections of the right and left coronary arteries and of the LAO cranial projection of the right coronary artery.

The results of this study demonstrate that one brief (median 20-minute) training session using a high-fidelity three-dimensional simulation module, when used to supplement traditional lecture-based learning can significantly accelerate novice or beginner trainees’ attainment of the competency to correctly identify coronary angiographic projections. Although the effect was modest, the amount of time investment involved relative to the gain in knowledge should be noted as it sometimes takes weeks to months to achieve the same level via traditional learning. Manipulating the C-arm in simulations allows the trainee to explore virtual three-dimensional coronary anatomies actively in real-time, thereby facilitating internal mental anatomical model construction, developing hand-eye coordination skills, and improving confidence in troubleshooting technical challenges in a safe learning environment. The ability to continuously track the coronary arteries in these simulation training sessions is a distinct advantage in visual-spatial learning compared to traditional interrupted 2-dimensional representation of coronary anatomy between shots in real world angiography.

Prior studies have consistently shown the greatest impact of simulation in novice trainees, consistent with findings reported from other simulation-based studies ^{9 16 20 21}. Correspondingly, the greatest improvement in our study was noted in ‘novice’ trainees, specifically residents, which are best representative of a new cardiology fellow with no prior cath lab experience. These findings further support the need for more studies to justify the adoption of a simulation curriculum early on in undergraduate and graduate medical education programs²².

The discrepancy between cardiology fellows and novice trainees is likely explained by the fellows’ previous attainment of the tested competency (basic anatomical identification on coronary angiograms) during their clinical training and experience. The study was conducted later in the academic year, and even our first-year cardiology fellows had already been exposed to coronary angiographic interpretation. Therefore, our described simulation training methodology should be integrated into a curriculum as part of introductory training. More broadly, our work demonstrates the importance of targeting a training protocol to the appropriate trainee. We speculate that cardiology fellows would best benefit from more advanced training protocols, such as teaching how to anticipate C-arm positioning to best visualize coronary anatomy.

To our knowledge, this is the first randomized controlled study to investigate the additive role of high-fidelity simulation training to traditional methods in teaching basic coronary angiography view interpretation to junior physicians. A recent study from France by Fischer et al²³ randomized 118 medical school students into simulation and traditional power-point based teaching. They reported that the simulation group did better in identifying coronary anatomy and coronary angiographic projections after a single simulation session. Although our main findings were similar, our study design is different. Firstly, we recruited a spectrum of trainees at various levels of clinical training, beyond medical students, to examine if more experienced trainees would benefit. Next, in order to allow for different learning speeds and preferences, subjects in the simulation arm were provided one-on-one instruction at the start of the exercise and then allowed independent unobserved practice time with no restriction on the amount of time spent on the simulator. Finally, since our subjects had varying amounts of exposure to clinical cardiology and familiarity with coronary angiography, we decided to focus on improvement in test performance from baseline, pre-intervention to post-intervention (delta scores) as our major primary outcome rather than an isolated post-intervention score by itself as reported by Fischer et al²³.

Limitations:

There are limitations to our study that are inherent with our sample size and study design. Our results for fellows are likely affected by their small sample size and varied amount of exposure to CA prior to the study. However, a study specific to cardiology fellows would require a multi-year and multi-center study, which would be limited by the general availability of coronary simulators. We did not perform quantitative assessment of baseline visuospatial skills of our study participants and so their influence if any on the study outcome is unknown. We were also unable to explore the effect of a single structured simulation session on long-term retention. Also, the additive effect of periodic booster training sessions on knowledge acquisition and retention was not studied. It would have been interesting to see if subjects in the control arm would have benefitted from crossing over to simulation training at the end of the study by administering a repeat assessment. This study was not blinded, but as the outcomes measurement was the performance on a multiple choice question test, the lack of blinding is unlikely to have caused bias.

In conclusion, this study highlights the growing role of simulation based medical education in the field of cardiology beyond just acquiring procedural skill in the cath lab. It suggests that even a single, brief targeted training session can rapidly improve novice or beginner trainees’ attainment of coronary angiographic projection competency. This study strengthens the case for the development of a framework for learning and competency assessment using simulation. Further large studies are needed to justify the cost of implementing simulation-based programs as part of cardiovascular fellowship training and should focus on tailored simulation training methodology for achieving specific competencies. Despite the cost barriers of integrating simulation in training, widespread adoption will hopefully result in a decrease in cost via the economies of scale.

Declarations: Of note, the manuscript submitted is original, with no portion under simultaneous consideration for publication elsewhere or previously published, only those who made important contributions to the study and are thoroughly familiar with the primary data are included as authors, and all authors are responsible for the contents and have read and approved the manuscript for submission to the BMC Medical Education.

Consent for publication: not applicable.

Conflict of Interest: Dr. Kwan Lee discloses he is a consultant for Mentice. The other authors have no conflicts.

Funding: None

Authors’ contribution:

Conception and design: KSL

Data acquisition: BN, WXW, WY, SK, IN

Data interpretation: KSL, KBK, DA, DF, OH

Drafting and revision: WV, JHI, JZL, KSL

Acknowledgements: The authors would like to thank Dr. Morton Kern for allowing us to use excerpts from his education video series titled “Cathlab Essentials with Dr. Morton Kern” in our study. We also appreciate the help of Jennifer Tsang, DO and Jonida Krate, MD for their assistance with recruitment of participants for the study.

Data Availability Statement: The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

DaRosa DA, Skeff K, Friedland JA, et al. Barriers to effective teaching. Academic medicine : journal of the Association of American Medical Colleges 2011;86(4):453-9. doi: 10.1097/ACM.0b013e31820defbe
Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236(4):458-63; discussion 63-4. doi: 10.1097/01.SLA.0000028969.51489.B4 [published Online First: 2002/10/09]
McGaghie WC, Issenberg SB, Petrusa ER, et al. A critical review of simulation-based medical education research: 2003-2009. Med Educ 2010;44(1):50-63. doi: 10.1111/j.1365-2923.2009.03547.x [published Online First: 2010/01/19]
Au K, Lam D, Garg N, et al. Improving skills retention after advanced structured resuscitation training: A systematic review of randomized controlled trials. Resuscitation 2019;138:284-96. doi: 10.1016/j.resuscitation.2019.03.031 [published Online First: 2019/04/01]
King SB, 3rd, Babb JD, Bates ER, et al. COCATS 4 Task Force 10: Training in Cardiac Catheterization. Journal of the American College of Cardiology 2015;65(17):1844-53. doi: 10.1016/j.jacc.2015.03.026 [published Online First: 2015/03/18]
Green SM, Klein AJ, Pancholy S, et al. The current state of medical simulation in interventional cardiology: a clinical document from the Society for Cardiovascular Angiography and Intervention's (SCAI) Simulation Committee. Catheterization and cardiovascular interventions : official journal of the Society for Cardiac Angiography & Interventions 2014;83(1):37-46. doi: 10.1002/ccd.25048 [published Online First: 2013/06/06]
Casey DB, Stewart D, Vidovich MI. Diagnostic coronary angiography: initial results of a simulation program. Cardiovascular revascularization medicine : including molecular interventions 2016;17(2):102-5. doi: 10.1016/j.carrev.2015.12.010 [published Online First: 2016/01/27]
Schimmel DR, Sweis R, Cohen ER, et al. Targeting clinical outcomes: Endovascular simulation improves diagnostic coronary angiography skills. Catheterization and cardiovascular interventions : official journal of the Society for Cardiac Angiography & Interventions 2016;87(3):383-8. doi: 10.1002/ccd.26089 [published Online First: 2015/07/23]
Bagai A, O'Brien S, Al Lawati H, et al. Mentored simulation training improves procedural skills in cardiac catheterization: a randomized, controlled pilot study. Circulation Cardiovascular interventions 2012;5(5):672-9. doi: 10.1161/CIRCINTERVENTIONS.112.970772 [published Online First: 2012/10/11]
Jensen UJ, Jensen J, Olivecrona G, et al. The role of a simulator-based course in coronary angiography on performance in real life cath lab. BMC medical education 2014;14:49. doi: 10.1186/1472-6920-14-49 [published Online First: 2014/03/14]
De Ponti R, Marazzi R, Ghiringhelli S, et al. Superiority of simulator-based training compared with conventional training methodologies in the performance of transseptal catheterization. Journal of the American College of Cardiology 2011;58(4):359-63. doi: 10.1016/j.jacc.2011.02.063 [published Online First: 2011/07/16]
Gurm HS, Sanz-Guerrero J, Johnson DD, et al. Using simulation for teaching femoral arterial access: A multicentric collaboration. Catheterization and cardiovascular interventions : official journal of the Society for Cardiac Angiography & Interventions 2016;87(3):376-80. doi: 10.1002/ccd.26256 [published Online First: 2015/10/23]
Jensen UJ, Jensen J, Ahlberg G, et al. Virtual reality training in coronary angiography and its transfer effect to real-life catheterisation lab. EuroIntervention : journal of EuroPCR in collaboration with the Working Group on Interventional Cardiology of the European Society of Cardiology 2016;11(13):1503-10. doi: 10.4244/EIJY15M06_05 [published Online First: 2015/11/09]
Popovic B, Pinelli S, Albuisson E, et al. The Simulation Training in Coronary Angiography and Its Impact on Real Life Conduct in the Catheterization Laboratory. The American journal of cardiology 2019;123(8):1208-13. doi: 10.1016/j.amjcard.2019.01.032
Prenner SB, Wayne DB, Sweis RN, et al. Simulation-based education leads to decreased use of fluoroscopy in diagnostic coronary angiography. Catheterization and cardiovascular interventions : official journal of the Society for Cardiac Angiography & Interventions 2018;91(6):1054-59. doi: 10.1002/ccd.27203
Voelker W, Petri N, Tonissen C, et al. Does Simulation-Based Training Improve Procedural Skills of Beginners in Interventional Cardiology?--A Stratified Randomized Study. Journal of interventional cardiology 2016;29(1):75-82. doi: 10.1111/joic.12257 [published Online First: 2015/12/17]
Norman G, Dore K, Grierson L. The minimal relationship between simulation fidelity and transfer of learning. Med Educ 2012;46(7):636-47. doi: 10.1111/j.1365-2923.2012.04243.x [published Online First: 2012/05/24]
Aeckersberg G, Gkremoutis A, Schmitz-Rixen T, et al. The relevance of low-fidelity virtual reality simulators compared with other learning methods in basic endovascular skills training. Journal of vascular surgery 2019;69(1):227-35. doi: 10.1016/j.jvs.2018.10.047 [published Online First: 2018/12/24]
Kern M. Cathlab Essentials with Dr. Morton Kern: Park Bench Media, 2011.
Gallagher AG, Ritter EM, Champion H, et al. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 2005;241(2):364-72. [published Online First: 2005/01/15]
Hsu JH, Younan D, Pandalai S, et al. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. Journal of vascular surgery 2004;40(6):1118-25. doi: 10.1016/j.jvs.2004.08.026 [published Online First: 2004/12/29]
Westerdahl DE. The Necessity of High-Fidelity Simulation in Cardiology Training Programs. Journal of the American College of Cardiology 2016;67(11):1375-8. doi: 10.1016/j.jacc.2016.02.004
Fischer Q, Sbissa Y, Nhan P, et al. Use of Simulator-Based Teaching to Improve Medical Students' Knowledge and Competencies: Randomized Controlled Trial. Journal of medical Internet research 2018;20(9):e261. doi: 10.2196/jmir.9634 [published Online First: 2018/09/27]

Table 1 Baseline characteristics

Characteristic	Control group (N=52)	Simulation group (N=53)	P value
Age (years)	30 ± 4	31 ± 4	0.28
Male gender	31 (60%)	28 (53%)	0.56
Visuo-spatial abilities:
Play sports	37 (71%)	29 (55%)	0.11
Play musical instruments	26 (50%)	21 (40%)	0.33
Play video games	25 (48%)	18 (34%)	0.17
Handedness
Right	44 (85%)	49 (92%)	0.15
Left	4 (8%)	0 (0%)
Mixed/ambidextrous	4 (8%)	4 (8%)
Preferred Learning Style
Visual	24 (46%)	32 (60%)	0.34
Auditory	2 (4%)	4 (8%)
Tactile	9 (17%)	7 (13%)
Read/Write	16 (31%)	10 (19%)
Stage of Training, N (%):
Med Student (YR 3&4)	11 (21%)	9 (17%)	0.99
PGY-1 Resident	12 (23%)	12 (23%)
PGY-2 Resident	12 (23%)	14 (26%)
PGY-3 Resident	9 (17%)	9 (17%)
PGY-4 CV Fellow	3 (6%)	4 (8%)
PGY-5 CV Fellow	3 (6%)	3 (6%)
PGY-6 CV Fellow	2 (4%)	2 (4%)
How confident are you with your knowledge on normal coronary anatomy?
1 (Not confident at all)	6 (12%)	6 (11%)	0.99
2 (slightly confident)	17 (33%)	16 (30%)
3 (somewhat confident)	21 (40%)	23 (43%)
4 (fairly confident)	8 (15%)	7 (13%)
5 (completely confident)	0 (0%)	1 (2%)
How would you grade your knowledge on coronary angiography?
Novice	15 (29%)	17 (32%)	0.87
Beginner	32 (62%)	29 (55%)
Intermediate	4 (8%)	6 (11%)
Advanced	1 (2%)	1 (2%)
Can you interpret coronary angiographic images accurately?
No	38 (73%)	31 (59%)	0.09
Somewhat	11 (21%)	21 (40%)
Yes	3 (6%)	1 (2%)

Table 2 Difference in Test Score after Control or Simulation Training According to Education Group

	All Training Arms	Control (N=52)	Simulation (N=53)^a
All education groups	4.6 ± 4.0	3.8 ± 3.7	5.4 ± 4.2
Medical Student (N=20)	4.1 ± 4.1	4.2 ± 4.5	4.0 ± 3.8
Resident (N=68)^b	5.1 ± 4.0	3.5 ± 3.4	6.6 ± 4.0
Fellow (N=17)	3.1 ± 3.8	4.4 ± 4.4	2.0 ± 3.1

^a p = 0.04 for training arm

^b p = 0.02 for interaction effect of training arm and education group, with residents showing the greatest positive impact of simulation.

Supplemental Digital Content 2 is not available with this version.

No competing interests reported.

SupplementalDigitalContent1Pretest.docx
Supplemental Digital Content 1. Word document. “Pre-test Coronary Angiographic Training Study”

A Randomized Controlled Trial of Simulation Training in Teaching Coronary Angiographic Views

Status:

Version 1

Abstract

Figures

Introduction

Methods

Study Population

Ethics

Study Design

Simulation Interface

Simulation Training

Didactic Teaching

Evaluation of Post-Training Performance

Statistical Analysis

Results

Study Population Characteristics

Test Performance Scores

Discussion

Limitations:

Conclusion

Declarations

References

tables

Supplementary File

Additional Declarations

Supplementary Files

Status:

Version 1