A total of 132 unique clinician participants consented to take part in the study across six JCs. Demographic information of participants is found in Table 2. Most participants were female (n=111, 84%) with 2 to 10 years of clinical experience (n=79, 59%), and were currently working in the inpatient acute setting (n=67, 51.5%). Of these participants, 79 (61.2%) completed baseline assessment, 63 (48.8%) completed the 10-month assessment and 47 (49%) completed the 16-month assessment. Two of the JCs did not participate in the 16-month assessment due to changes in clinical services provided during the COVID-19 pandemic, resulting in their JCs requiring an extended break after 12 sessions. In addition, four research mentors completed the support audit (median years of post-doctorate experience=4.5 years, range 4 to 11 years).
Table 2. Demographics of participants
Demographics
|
N (%) 132 unique participants
|
Gender
|
|
Male
|
21 (15.9%)
|
Female
|
111 (84.1%)
|
Age (years)
|
|
20-29
|
71 (53.8%)
|
30-39
|
41 (31.1%)
|
40-49
|
16 (12.1%)
|
50-59
|
4 (3.0%)
|
Profession
|
|
Dietician
|
3 (2.3 %)
|
Pharmacist
|
18 (13.6%)
|
Dentist
|
18 (13.6%)
|
Psychologist
|
1 (0.8%)
|
Occupational Therapist
|
45 (34.1%)
|
Speech Pathologist
|
45 (34.1%)
|
Physiotherapist
|
1 (0.8%)
|
Allied Health Assistant
|
1 (0.8%)
|
Setting
|
|
Community setting
|
17 (13.1%)
|
Hospital Outpatient
|
7 (5.4%)
|
Inpatient Acute
|
67 (51.5%)
|
Multiple
|
28 (21.5%)
|
Clinical Education
|
3 (2.3%)
|
Inpatient Mental health
|
1 (0.8%)
|
Clinical experience (years)
|
|
Less than 2
|
26 (19.4%)
|
2-5
|
41 (30.6%)
|
5-10
|
38 (28.4%)
|
10-15
|
13 (9.7%)
|
Greater than 15
|
16 (11.9%)
|
Higher Research Degrees
|
|
None
|
46 (60.5%)
|
Masters of Research
|
2 (2.6%)
|
PhD
|
1 (1.3%)
|
Honours
|
12 (15.8%)
|
Masters - Other
|
7 (9.2%)
|
Graduate Diploma
|
5 (6.6%)
|
Post Graduate Certificate
|
3 (4.0%)
|
Have attended EBP Training
|
53 (40.1%)
|
Number of journal clubs attended
Mean (SD)
|
5.72 (3.83)
|
Completed Assessment at baseline
|
79 (61.24%)
|
Completed Assessment 10-months
|
63 (48.84%)
|
Completed Assessment 16-months
|
47 (48.96%)
|
EBPQ and Adapted Fresno Test
As shown in Table 3, on the EBPQ participants self-reported improvements in their EBP confidence at 10- and 16-months compared to baseline. Minimal changes were apparent in EBP attitudes and EBP use across time points. On the Adapted Fresno Test EBP skills were found to be improved at 10 and 16-months.
Journal Club Culture Questionnaire
Clinician ratings of their manager’s expectation for them attend JC and their access to expertise and resources outside the team were found to improve compared to baseline at 10-months, with the latter continuing to improve 16-months following JC participation (see Table 3). An increase at 16-months compared to baseline was also found for information sharing within the team. At the 10-month and 16-month time points, the average ratings of agreement to wanting the JC to continue and recommending it to others were approaching 4 out of 5 (mean= 3.7 and 3.9 respectively), indicating general positive agreement with these statements.
When covariates were taken into consideration, prior EBP training was found to have an effect on the Adapted Fresno Total scores, with adjusted changes from baseline to 10 months being 6.4 (95% CI: 0.23- 12.5, p=0.042) and baseline to 16 months 7.8 points (95% CI, 1.09, 14.9, p=0.023). Gender and total attendance were found to have a significant effect on the group participation item of the JCC questionnaire leaving a negligible effect of time-point at 10 months; 0.01 (95% CI: -0.17, 0.19, p= 0.90) and 16 months; 0.11 (95% CI -0.06, 0.29, p=0.37) compared to baseline.
Table 3: Results of EBPQ, Adapted Fresno Total, and Journal Club Culture Questionnaire scores at baseline, 10-months and 16-months.
|
Baseline/Pre
Mean (SE)
|
10month
Mean (SE)
|
16-months mean (SE)
|
Pre-10 month mean difference and 95% CI, p-value
|
Pre-16 month mean difference and 95% CI, p-value
|
EBPQ & Adapted Fresno
|
EBP use (Q1 total, max score= 42)
|
23.44 (1.20)
|
25.54 (1.25)
|
24.32 (1.33)
|
2.2 (0.44, 4.1) 0.015
|
0.9 (-1.1, 2.9) 0.40
|
EBP attitudes (Q2 total, max score= 28)
|
21.69 (0.45)
|
21.21 (0.48)
|
21.80 (0.53)
|
-0.41 (-1.3, 0.52) 0.39
|
0.22 (-0.81, 1.2) 0.68
|
EBP confidence (Q3 total, max score =98)
|
56.94 (1.79)
|
61.82 (1.87)
|
62.62 (1.98)
|
4.9 (2.2, 7.5) 0.0003
|
5.7 (2.7, 8.7) 0.0002
|
Adapted Fresno Test Total
(max score= 168)
|
70.38 (5.59)
|
76.96 (5.74)
|
78.19 (5.94)
|
6.6 (0.43, 12.7) 0.036
|
7.8 (0.85, 14.7) 0.028
|
JCC Questionnaire
|
Use of EBP valued by team
|
4.47 (0.11)
|
4.56 (0.11)
|
4.61(0.12)
|
0.09(-0.06, .24) 0.22
|
0.14 (-0.03, 0.31) 0.10
|
Manager expects attendance
|
4.02 (0.34)
|
4.28 (0.34)
|
4.19 (0.34)
|
0.26 (0.04, 0.48) 0.02
|
0.17 (-0.07, 0.42) 0.16
|
Sense of JC ownership
|
3.90 (0.15)
|
3.74 (0.16)
|
4.00 (0.17)
|
-0.16 (-0.38, 0.06) 0.16
|
0.10 (-0.14, 0.34) 0.42
|
JC is applicable to practice
|
4.42 (0.15)
|
4.13 (0.16)
|
4.11 (0.16)
|
-0.29 (-0.48, -0.10) 0.002
|
-0.31 (-0.52, -0.10) 0.004
|
JC attendance is a priority
|
3.97 (0.28)
|
3.83 (0.29)
|
3.72 (0.29)
|
-0.14 (-0.35, 0.07) 0.19
|
-0.25 (-0.48, 0.02) 0.035
|
JC is of benefit to me
|
4.26 (0.16)
|
4.08 (0.16)
|
4.02 (0.17)
|
-0.19 (-0.37, -0.00) 0.05
|
-0.25 (-0.46, -0.04) 0.02
|
Group participation valuable part of JC
|
4.43 (0.09)
|
4.46 (0.09)
|
4.55 (0.10)
|
0.06 (-0.12, 0.24) 0.50
|
0.16 (-0.03, 0.35) 0.10
|
Sharing of knowledge and skills within team
|
3.74 (0.16)
|
3.90 (0.17)
|
4.06 (0.18)
|
0.15 (-0.07, 0.38) 0.17
|
0.32 (0.07, 0.57) 0.01
|
Access to expertise and resources outside team
|
3.46 (0.09)
|
3.75 (0.11)
|
4.02 (0.12)
|
0.29 (0.05, 0.54) 0.02
|
0.56 (0.29, 0.83) 0.00005
|
Think JC should continue
|
N/A
|
3.76 (0.17)
|
3.74 (0.17)
|
-0.02 (-0.29, 0.24) p= 0.87 (10 – 16-months)
|
Recommend JC to others
|
N/A
|
3.91 (0.17)
|
3.97 (0.17)
|
0.065 (-0.18, 0.31) p= 0.607 (10 – 16-months)
|
N.B: Mean= model-based estimate of mean score. SE= standard error for model-based estimate.
Qualitative responses of Journal Club and Culture questionnaire
Five categories were identified in the questionnaire responses: 1) enablers and 2) barriers to EBP culture, 3) enablers and 4) barriers to JC; as well as 5) suggestions for improvement. Most commonly reported categories and subcategories are shown in Table 4 for both 10 month and 16-month time points. In general, responses did not conceptually differ across time points. Some barriers to JC, including lack of preparation due to time constraints and variable attendance, that were evident at the 10-month time point were not reported at the 16-month time point. The most commonly reported enablers to EBP culture were team educational opportunities (with “Journal club” being the most frequently reported of these opportunities); and having collaborators and people as resources including “EBP champions” and “Research Fellows”. The most predominant barrier was time and competing caseload demands of clinicians, “Time constraints to research EBP. Competing clinical and non-clinical priorities” (P089), as well difficulties implementing EBP as described as “possible lack of confidence in integrating EBP into clinical practice if there is a change required” (P027).
Enablers to JC related to structural components of the format included choosing clinically relevant topics, “having a team consensus on choosing articles relevant to the group” (P119) and active participation from the group, with one clinician commenting, “Group participation is important for the success of journal club” (P063). Supportive team factors were also commonly reported as enablers, including “Sharing between all team members “(P106) and “Open mindedness and respect of differing opinions” (P128). Conversely, barriers to JC included topics not being relevant, lack of active participation or preparedness and general time constraints, for example “time constraints in preparing for JC i.e. pre-reading article.” (P005).
Table 4: Responses for JCC open ended questions – 10-months
Categories and subcategories
|
No. of mentions 10-months
|
No. of mentions 16-months
|
|
Team educational opportunities
|
|
|
Professional Development in clinical skills/ in-services
|
12
|
15
|
Individual learning time
|
9
|
1
|
Quality projects and portfolios
|
7
|
6
|
Regular supervision
|
8
|
9
|
Encouragement of CPD courses
|
6
|
6
|
Journal club
|
19
|
29
|
Team discussions EBP literature and clinical cases
|
12
|
4
|
Collaborators & People resources (e.g., EBP champions, Research Fellows)
|
14
|
18
|
Supportive workplace culture that values EBP
|
15
|
10
|
Supportive managers and seniors
|
7
|
9
|
Protected time
|
8
|
1
|
It's an expectation
|
7
|
4
|
Accessible resources
|
6
|
5
|
2. Barriers of EBP culture
|
|
|
Time and caseload demands
|
40
|
37
|
Difficulties with EBP implementation
|
4
|
14
|
Personal and team reduced knowledge & skills
|
5
|
4
|
Type and quality of research
|
3
|
2
|
Staffing issues
|
4
|
1
|
Lack of EBP value or commitment
|
1
|
4
|
3. Enablers of Journal Club
|
|
|
Structure
|
|
|
Having an academic present or support available
|
11
|
8
|
Set regular times
|
6
|
4
|
Protected time
|
6
|
9
|
Clinically relevant topics
|
11
|
19
|
Early circulation of articles
|
2
|
10
|
Preparation of presenter and facilitator
|
5
|
4
|
Specific roles allocated and shared
|
6
|
9
|
Supportive team factors
|
16
|
9
|
Supportive leadership, managers and seniors
|
6
|
7
|
Research and clinical knowledge
|
7
|
3
|
Attendance and active participation
|
19
|
12
|
4. Barriers of Journal Club
|
|
|
General time constraints
|
8
|
20
|
Irrelevant topics
|
7
|
12
|
Personal factors such as fatigue, motivation, stress
|
6
|
5
|
Variable attendance
|
5
|
|
Staffing impacting attendance
|
11
|
2
|
Lack of active participation or preparedness
|
7
|
11
|
Reduced knowledge, skills & confidence
|
10
|
6
|
Lack of preparation generally due to time constraints
|
22
|
|
5. Suggestions for improvement
Increased education and support
Structure changes
Scheduling changes
Changes to preparedness
Topic selection suggestions
|
19
14 7
7
3
|
7
18
3
3
11
|
Influence on clinical practice
Across the 16 sessions, 64 clinicians reported that they updated guidelines, processes or pathways as a result of JC participation, 88 clinicians reported adopting new treatment strategies or resources, 30 clinicians reported starting new quality projects and 6 clinicians reported starting new research projects. One clinician also reported discontinuing a current practice due to lack of evidence as appraised in a JC session (See Supplementary file 4).
Treatment fidelity
As depicted in Figure 1, most components of the TREAT format were adhered to across sessions. Components reported to have the highest adherence (100%) were having a relevant topic, a facilitator present, group appraisal and application to clinical context discussed. Items with lowest adherence were reviewing actions from the minutes (38.1%) and seeking library support (51.8%). The level of support provided across sessions by research mentors is shown in Table 5. The first two sessions were all facilitated by the research mentor, while in subsequent sessions, the level of support varied across JCs. However, a general pattern of reduced support as the sessions progressed was evident in all JCs.