Study Design: This single-blinded randomized controlled trial utilized a 1:1:1 allocation ratio with block randomization to one of the two pre-course intervention arms (NRP eSim™ virtual simulation plus NRP textbook study or NRP textbook study alone) and to one of the three post-course arms: every-2-months refresher training with NRP eSim™ virtual simulation (eSim™ group),6 every-2-months review of a video demonstrating a complex resuscitation (video group), or no structured refresher (control).
Inclusion Criteria
All pediatric healthcare professionals who enrolled in an NRP course were eligible to participate.
Participants were designated as either novices (first-time NRP training) or experienced (at least one prior NRP training).
Exclusion Criteria
Pediatric healthcare professionals who were unable to complete 6 month follow up test/simulation (e.g. graduating resident/fellow).
Setting
Trial sites were at four academic institutions located in the Midwest, West and Southern part of the United States. (University of Washington, Seattle, Washington, Stanford University School of Medicine, Palo Alto, California, University of Texas Southwestern, Dallas, Texas, and Saint Louis University, St. Louis, Missouri). The study was approved by the Institutional Review Board at each participating site.
Interventions
In the NRP eSim™, learners had access to five screen-based simulations, which included a mix of term and preterm neonatal resuscitation scenarios. Access to the online NRP Textbook and other reference materials were available within the scenarios. At the start of the virtual simulation, the learner was presented with the case information and asked to set up the equipment. During the scenario, the learner ‘dragged and dropped’ appropriate equipment over the infant to simulate the resuscitation steps (Supplemental Fig. 1).
Before the course, participants were randomized to two arms of preparation: NRP textbook study only and NRP study textbook plus eSim™. Participants then attended a standard NRP course. After the NRP course the groups utilized their randomly assigned refresher. One group was assigned to no structured refresher training. The second group was assigned to review a 10-minute video developed specifically for this study reviewing the steps of the 7th edition neonatal resuscitation algorithm including initial steps of resuscitation, positive pressure ventilation, endotracheal intubation, chest compressions, intravenous (IV) epinephrine adminstration, and a normal saline bolus via an emergently placed umbilical venous catheter. The third group was assigned to complete an NRP eSim™ case featuring a term newborn requiring resuscitation. Both refresher intervention groups (video group and eSim™ group) were sent reminders at 2 and 4 months after their NRP course with links to access the video or the eSim™ platform, depending on their group assignment (Fig. 1). Online access to the video and NRP eSim™ was password protected through the NRP learning platform.
Outcomes: The primary outcome was time to administration of IV epinephrine during a live, simulated newborn resuscitation conducted six months after the participant completed their NRP training. Secondary outcomes included scoring various NRP skills during the simulated resuscitation (Supplemental File 1: NRP Team Leader Performance Scoring Sheet). The NRP skills performance score was developed as a scoring tool by the study team and includes 25 unique resuscitation steps with a scale from not done, partially correct, and completely correct for each step. The tool was designed to focus on the study participant leading the resuscitation and did not evaluate any performance on the part of the embedded, confederate team members. Simulation event times and performance were compared to baseline times and performance from simulations conducted immediately following NRP training. Simulations used a standard case prompt, high-fidelity newborn manikin, and equipment (Supplemental File 2). The participant was instructed to lead the resuscitation team which included embedded personnel who responded to the study participant’s questions and requests. The manikin’s vital signs were programmed to follow a standard timeline with heart rate and oxygen saturations not recovering until placement of an endotracheal tube, provision of chest compressions, and administration of IV epinephrine. All simulated resuscitations were video recorded for review by study authors with extensive experience who were blinded to the participant's demographics and group allocation. Each video was reviewed for the time required to complete key NRP algorithm steps and scored using a checklist. Video reviewers received training on the scoring checklist. All reviewers rated performance on a set of training videos to ensure consistency before scoring participant videos.
Sample Size
Sample size was calculated using an assumed mean of 330 +/- 45 seconds between birth and the administration of IV epinephrine16,17. We aimed to show a 30-second difference in time to IV epinephrine and assumed a 15% dropout rate between NRP training and six-month follow-up. The goal sample size was 72 people per group. Due to higher-than-expected dropout rates, the initial recruitment goal was increased from 248 to 315 participants. Recruitment closed at 306 participants in March 2020 due to the COVID-19 pandemic and the inability to complete live simulations.
Randomization
Participants were randomized to 3 allocation groups stratified by experience (novice vs. experienced) and profession (physician vs. non-physician). Randomization was performed online (randomizer.org) using blocks of 4, and allocations were printed and placed in sequentially numbered opaque envelopes at each site by personnel not on the study team. Study team members running the simulations were blinded to the allocation group, and NRP staff notified participants of their assignment via email. Video reviewers were blinded to both allocation groups and whether the video was immediately post-NRP training or from the six-month follow-up simulation.
Statistical methods
All study data was collected and stored in REDCap. Statistical analyses were performed using SAS version 9.4 (SAS Institute, Cary, North Carolina). P-values < 0.05 were considered statistically significant. Baseline demographic data for the groups were compared using chi-square for categorical characteristics and one-way ANOVA for continuous data. We used one-way ANOVA to examine associations of the time to each action and performance scores for NRP between the 3 arms. Primary analyses used the intention-to-treat principle. The Kruskall Wallis test was used for a secondary comparison of performance scores based on the trajectory of performance between the initial and follow-up simulation. Additional secondary analysis included examining performance scores based on self-reported utilization of refreshers was done using Kruskall Wallis tests.