Pilot-testing of the on-line training – Pre-Post Simulation Study
Of the 18 nurses who indicated initial interest, 13 consented to the study and participated in the pre-training simulations and 12 completed the training modules and completed the post-training simulations. Primary reasons for non-participation were a lack of time to schedule the simulations and concerns related to being evaluated in simulations. Participants were predominately female (n=11) and experienced (>5 years) in oncology (n= 9). Although no formal screening programs had been implemented at either hospital, five participants indicated screening for anxiety and depression was part of their current role, ten nurses reported their role encompassed referral of patients for psychosocial support and four nurses reported their role included provision of psychosocial support.
Assessing the Effects of Training
Table 2 presents the means and effect sizes for pre and post-training communication skills scores and consultation length. Inter-rater reliability for pre-training scores ranged from .566 (moderate) to .935 excellent and for post-training scores ranged from .859 (good) to .952 (excellent). Across the three consultations, communication improved from pre- to post training for all participants, although only making a referral demonstrated a large and significant within-group improvement between pre and post training (g=.83). Incorporation of the skills acquired through the training did not increase the length of consultations (Table 2), and there was a large and significant within-group decrease in consultation length when making a referral post-training (g=1.17).
Confidence Ratings
Total self-assessed confidence increased from a pre-training mean score of 14.7 (SD 4.4) to 16.0 (SD 3.7) post-training. This increase was also observed for all tasks assessed, although only identifying anxiety symptoms was statistically significant (t=2.2, p<.05). Scores for the additional post-training item indicated 63% (n=7) participants were confident applying the ADAPT CP in their local institution. Mean item scores are listed in Table 3.
Acceptability
Qualitative feedback confirmed that the content of the online training was appropriate for oncology health professionals, and the format acceptable and engaging.
I enjoyed getting the information from three different aspects, from the social worker, the psychiatrist, and psychologist… because they gave you a bit more information about how to approach thing…. I thought the scenario-based way it was set up was good (UPN3).
I loved the fact that it did guide you in the decision-making of the level of assistance [stepped care] that is required for each person. … I thought some of the skills with communication were excellent (UPN6).
A few participants noted there was a lot of content and finding time to complete the training was challenging.
I think all of it was very good information, there was just a lot of it (UPN4)
it was probably… way too long. 4 hours is a lot of time for people if you’re trying to do it during work. (UPN 2).
More experienced participants perceived that the training would be of most benefit to those with less oncology experience.
I think a lot of it I’ve done before myself, but I can imagine if you were in a different situation, if you’d just come into oncology, or you were on the wards, if you were not very experienced staff [the training would be helpful (UPN2)
Overall participants viewed the training as positive and perceived they increased their knowledge and skills.
It was quite an empowering 5 modules … I found the level of information, the videos, the short questions, actually quite useful. You think you’re doing it but when you do the questions, you can see how you can do better, you see how it can impact on a person. It made me identify the signs and symptoms of anxiety and depression a lot more than perhaps I would have prior. (UPN16)
The qualitative data was supported by the post-training acceptability survey (Table 4), with 81.8% (n=9) of participants indicating the information was practical and useful, enabled them to reflect on their practice (75%) and taught them additional skills (72.7%).
Real world Acceptability – uptake into clinical practice
Of the 286 active users registered within the ADAPT portal, 37 participants (12.9%) from across 11 of the 12 sites accessed the training via the link from the ADAPT portal. However only 20 of 37 (54%) of participants who accessed training completed the training modules, and only 4 of 37 (11%) completed the post-training assessment. Participants who accessed training included nurses (n=13), social workers (n=4) and clinical administration staff (n=2), with two participant disciplines not reported. Forty-seven percent (n=9) of those who accessed training indicated they had less than two years in oncology, although 42% (n=8) of participants reported more than 10 years oncology experience. No data is available for those registered in the portal who did not access the training.
Analysis of the page clicks (n=96) confirmed that participants accessed the training on multiple occasions throughout the ADAPT CRCT suggesting that the training was not completed in a single session but was accessed on an as needs basis.
Confidence Ratings
Total self-assessed confidence increased from a pre-training mean score of 14.4 (SD 45.2) to 17.0 (SD 5.2) post-training. This increase was also observed for all tasks assessed, although the small number of post-training responses (n=4) disallowed further analysis. Mean item scores are listed in Table 5.
Acceptability
Qualitative feedback exploring staff perceptions of the resources more broadly was obtained at three timepoints within the ADAPT CRCT (baseline (T0), 6 months (T1) and 12 months (T2) post implementation) and confirmed participants who accessed the training perceived it to be acceptable, appropriate and informative. These interviews also confirmed that access continued for the 12 months implementation period rather than being limited to the study initiation phase.
Oh, the eviQ stuff. Yeah, that was really good. I liked doing it actually, it just clarified a lot of things and gave me that reminder - - - about why we’re doing it, and also how to initiate it and bring it up in conversation and, – I mean, you’ve been nursing for so long and not incorporating it all that often. Like, it’s, it’s hard to change your practice --- so it was really helpful to actually get those skills and go through the scenarios as to how to incorporate screening and introducing it. (Nursing Staff, T0, S12PID01)
I found the resources ADAPT provided were really helpful to start off with, it … confirmed and enhanced your knowledge and experience in the areas. …I watched some of the… webinars... On EviQ. They were really helpful. (Nursing Staff, T1, S10PID02)
The online training was great. I did do quite a lot of it, and - - - I found it really wonderful. I highly recommend that side of the training. (Psychosocial Staff, T2, S07PID05).
Other reported barriers to uptake included preference for face-to-face training, or a lack of protected time to undertake the online modules during work hours.
I guess, different people have preference for different methods of learning but, … you could allocate 30 or 45 minutes… where people go to the education room and just have protected time to sit for that time and do it. …This idea that people would just do it in their own time online often means it doesn’t happen. So I think it’s certainly good and important to have it available, um, but, – the challenge is how to do it. (Medical Staff, T1, S11PID05)