Online training in evidence-based medicine and research methods for GP Registrars: a mixed-methods evaluation of engagement and impact

doi:10.21203/rs.3.rs-258005/v1

Download PDF

Research Article

Online training in evidence-based medicine and research methods for GP Registrars: a mixed-methods evaluation of engagement and impact

https://doi.org/10.21203/rs.3.rs-258005/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Evidence-based medicine (EBM) is a core skillset for enhancing the quality and safety of patients’ care. Online EBM education could improve clinicians’ skills in EBM, particularly when it is conducted during vocational training. There are limited studies on the impact of online EBM training on clinical practice among GP registrars (trainees in specialist general practice). We aimed to explore GP registrars’ experience with an online interactive course that was a compulsory component of their vocational training. The course was developed by content-matter experts with educational designers to encompass effective teaching methods (e.g. it was interactive and used multiple teaching methods).

Methods

Mixed-method data collection was conducted after individual registrars’ completion of the course. The course comprised six modules that aimed to increase knowledge of research methods and application of EBM skills to everyday practice. GP registrars who completed the online course during 2016–2020 were invited to complete an online survey asking about their experience and satisfaction with the course. Those who completed the course within the six months prior to data collection were invited to participate in semi-structured phone interviews. A thematic analysis approach was used to analyse the data from qualitative interviews.

Results

The data showed the registrars were generally positive towards the course and the concept of EBM. They stated that the course improved their confidence, knowledge, and skills and consequently impacted their practice. The students perceived the course increased their understanding of EBM with a Cohen’s d of 1.6. Registrars identified factors that influenced the impact of the course. Of those, some were GP-related including their perception of EBM, and being comfortable with what they already learnt; some were work-place related such as time, the influence of supervisors, access to resources; and one was related to patient preferences.

Conclusions

This study showed that an online education program was able to improve GP registrars’ knowledge, confidence, skill and practice of EBM over the period of three months. It also highlights the supervisor’s role on GP registrars’ ability in translating the EBM skills learnt in to practice and suggests exploring the effect of EBM training for supervisors.

Educational Philosophy and Theory

evidence-based medicine

online education

clinical practice

mixed methods

Evidence-based medicine (EBM) is the integration of best available clinical evidence with clinical expertise and patient values to inform a clinical decision (1). It is recognised as a core skillset for improving the quality and safety of health care (2). This is acknowledged by the inclusion of EBM competencies in the curriculum for general practice education throughout the training continuum - from undergraduate learners to continuing professional development programs for established GPs (3,4).

Despite some evidence that GPs generally acknowledge the importance of EBM (5), it is not always practiced in routine care (6). For instance, Australian GPs and GP registrars (trainees in specialist general practice) have been observed to prescribe antibiotics for respiratory tract infections in a non-evidence-based manner (7,8).

There are a range of influences on the use of EBM in practice. Many of these factors are shared among GPs and GP registrars. Some of these barriers are at the GP-level, such as doubt about the applicability of the research evidence to practice (9,10)) or lack of skills in finding, appraising and applying evidence (5,6,11–13). Some are workplace-related such as the influence of previous practice or peers (9) and resource constraints (e.g. time pressure, access to resources such as reliable internet or subscriptions to sources) (5,6,11–14). Patient factors also influence clinical practice. While patient-centred care is an important feature of high-quality care (15), navigating between EBM and patient preferences and beliefs for treatments that are unsupported by evidence, can be challenging (5,6,11–13). In addition, there are some specific barriers related to GP registrars, for example, GP registrars often seek answers for their clinical questions, especially for more complex ones, (16) by consulting supervisors and colleagues (17). This could be a barrier or facilitator to the practice of EBM, depending upon the supervisor (18,19), Some studies also reported that the atmosphere established by supervisors (authoritarian vs collaborative) could be a barrier for GP registrars (20).

Medical education has a major role in preparing a workforce skilled in EBM, but its translation from learning to practice is challenging (21,22). Specific EBM teaching strategies such as clinically integrated training, problem-based learning, and e-learning can improve knowledge, attitudes, and skills in undergraduate medical students (21). However, there is limited evidence that these strategies directly influence clinical practice (21).

Internet-based learning activities offer a range of advantages, such as ease of access from various settings and for a large audience (23). Online training can be efficient compared to the traditional face-to-face methods as it allows distance learning when local training opportunities and resources are limited; provides convenience and flexible learning; reduces travel time and expenses for learners (24). It can be beneficial and time-efficient when trainees have diverse ranges of background knowledge and customised course content is needed (25). These features make online training a pragmatic method for training medical professionals in full-time clinical practice. A recent review of 14 studies concluded that online training can be as effective as other alternative methods for increasing knowledge and improving clinical practice of medical professionals (23).

While online learning has potential benefits, it must engage and empower learners for the learnings to be applied in the real world. Boettcher’s review of pedagogical theory and research identified the following core principles for effective teaching (traditional and technology-enhanced): that teaching should be interactive, customised to learners’ background knowledge, deliver information in the form of organised chunks, and provide opportunities for learners to write, explain and analyse (26). EBM educational interventions that use a variety of learning methods (e.g. video, written materials) are more likely to have a significant effect on learning EBM than those that used a single method (27,28). More broadly, training should be relevant to practice (29), case-based (30,31) and implemented when learners are exposed to clinical cases to practice the skills learnt (29).

The study objectives

This research explored GP registrars’ experience with an online interactive training course that aimed to increase knowledge of research methods and by GP registrars and, ultimately, produce practitioners who can use evidence in practice. The findings of this study not only can be used for the improvement of the current training course but also for informing future similar course development by identifying its efficacy, strengths and limitations. The study’s research questions were:

What were the GP registrars’ experiences with the course (including how engaged were they with the course?)
What were the impacts of the course on GP registrars’ attitudes, knowledge, skills and clinical practice?
What factors influenced the impact of the course on the GP registrars’ clinical practice?

Design

The evaluation was a one-group design (post-test only) conducted after completion of the course, using a triangulation mixed methods design.

Recruitment and sample

All GP registrars who completed the online course during Nov 2016-Feb 2020 (n=1142) were invited to complete an online survey via a link within the course.

All GP registrars who completed the course within the six months prior to data collection (2019-2020) were invited to participate in semi-structured interviews. Registrars were invited to express interest in being interviewed for the study via:

An invitation within the online survey
An invitation from GP Synergy, sent to registrars on completion of the course and to registrars who completed the course since 1^st January 2019
An invitation within the online Research Skills Module.

Sampling was purposive, based on potential participants’ characteristics of medical training (Australia vs overseas) and GP training pathway (general vs rural).

Intervention: “the course”

An online education course (“the course”) was designed and built for GP registrars to complete as a compulsory component of their vocational specialist training. This was commissioned by GP Synergy Ltd, the vocational GP regional training organisation in New South Wales and the Australian Capital Territory, Australia. The GP regional training organisations in Australia are geographically-based, government-funded, not-for-profit entities. The Australian GP training program has an annual intake of approximately 1500 GP registrars nationwide, of which approximately one-third are trained by GP Synergy (32). The course was developed by content and education experts from UNSW Australia (including academic GPs) and an interactive software developer (Smart Sparrow Pty Ltd) to ensure modules were educationally sound.

The course was designed to provide a grounding in the research, epidemiology and critical evaluation and to be consistent with the knowledge required to practice as an evidence- based practitioner. The overarching aim of the course was “to have registrars using evidence and practising EBM in their everyday practice” (33). It comprised six modules of critical thinking and ethical principles in human research, critical appraisal skills, exploring the evidence on a clinical question including systematic review and meta-analysis, clinical epidemiology, quantitative research methods and qualitative research methods. Each module was divided into three-to-six lessons with lesson-specific learning objectives (see Additional file 1, Table S1). The course aimed to enable participants to:

identify links between research and practice
apply the research findings to examples from practice
describe the principles of research design
participate in research.

The modules included text with visual learning methods such as graphs and images, video lectures, links to publicly accessible video resources, interactive exercises (e.g. multiple choice), clinical scenarios and further reading. Each module was expected to take about an hour to complete, not counting additional reading.

Instruments and data collection

The online survey asked participants to indicate the degree to which they agreed with 11 statements that described their experience with the course using a 6-point rating scale; rate their understanding of the topic before the course and after completing the course on a scale from 0 to 10; and answer two open-ended questions: 1) what they liked most about the course and 2) what they would like to see changed. Participants self-completed the online survey via the link provided in the course.

The semi-structured interview schedule (Box 1) contained two sections:

Experience with the course
Impact of the course on clinical practice.

Interviews were conducted by telephone in one or two sessions, depending on the course completion time:

Two sessions: If the Registrar had completed the course less than one month before recruitment, the first interview was restricted to section one of the interview schedule. A second interview to complete section two of the interview schedule was conducted three months later.
One session: If the Registrar had completed the course at least three months before recruitment both sections of the interview schedule were completed in the one interview.

None of the participants completed their first interview between one and three months after completing the course. The timing was designed to allow participants the opportunity to apply the knowledge and skills gained during the course before answering questions about how the course influenced clinical practice.

The interviews were conducted via telephone by a trained Research Officer [HR-A] who had not been involved in the course development. The average times to complete the interviews were 23 minutes (first interviews) and 19 minutes (second interview).

Box 1 Interview questions

1. What are your thoughts on the program?

2. If you didn’t complete the program, can you provide feedback on why?

3. How engaging were the online modules?

4. How easy was it to use the online modules?

5. Was the content pitched at the right level of difficulty and detail?

6. Were your learning needs and expectations met?

7. How relevant were the topics to you as a general practitioner?

8. We are keen to determine whether you are using the EBM skills learnt in the modules in your practice.

9. Has completing the program led to you considering further training in research

Data analysis

Descriptive statistics were used to present quantitative data from the online survey. The difference between the self-report level of understating of the topic before and after the course was tested using paired t-test and the effect size compared using Cohen’s d. Data analysis was performed using SPSS (version 23). The answers to open-ended questions were coded and summarised. These were then discussed between authors [CS, HR-A] and grouped into broader categories and sub-categories based on their similarities and differences.

Data collection and analysis were concurrent and iterative for interviews. The interviews were transcribed verbatim and coded using the software program NVivo (Version 12 Pro). A thematic analysis approach was used (34) to analyse the data from qualitative interviews. The initial few transcripts of both the first and second interviews were open coded by pairs of researchers and the codes compared and discussed among five authors [CS, MH, HR-A, CT, PM]. Later codes were updated based on the discussion, and the rest of the transcripts were coded by one of the authors [HR-A]. In all steps of coding, the analyst constantly refined the earlier codes in the light of newly emerged codes. When all data were coded, the generated themes and codes were again discussed among all authors, until the consensus is reached. we aimed to achieve thematic saturation but due to Covid-19, data collection was ceased before the data saturation was complete. There was still good saturation in most of the themes.

To ensure trustworthiness, an audit trail of all the steps and decisions made during the research process were recorded (35). The researcher triangulation method (35) was used by involving multiple analysts to bring different perspectives into the findings. Verbatim quotations of various participants were provided to support the study findings. Considering reflexivity, most of the authors (NZ, MH, CT, PM, TL) were academic GPs in current clinical practice, and all authors except one (HR-A) were experienced and involved in activities relevant to teaching EBM to GPs. All authors had prior experience in analysing qualitative data. The main analyst [HR-A] kept a reflexivity journal during analysis to discuss it with all authors.

When data analyses were completed, qualitative and quantitative findings were synthesised to answer the study questions. Codes were reviewed for patterns, and they were further revised and developed into themes to answer the research questions. The final themes are presented in a figure in the result section.

Ethics

Ethical approval was obtained from the Human Research Ethics Committee at UNSW (Approval number HC16483), and participants provided informed consent prior to participation in the study.

Participants

Of 1142 GP registrars who invited to take part in the online survey, 391 (34.2%) completed the survey between Nov 2016-Feb 2020. Fourteen GP registrars participated in interviews. Of the ten participants for whom follow-up interviews were indicated, eight completed the second interview. Details of the recruitment process and reasons for exclusion are provided in Figure 1.

The interview sample comprised nine women, five men; two registrars enrolled in the rural pathway, and only one registrar who had completed medical training overseas. The interviewees enrolled in the course during the second or last year of their three-year (full-time equivalent) program. Participants stated that they spent an average of nine hours to complete the whole course.

Results are presented to answer each research question.

RQ1. Participant experience with the course

Registrars’ responses to 11 questions on satisfaction with the course are presented in Figure 2.

Data from the online survey and the interviews indicated that participants’ course experience was generally positive. As illustrated in Table 1, the triangulated methods provided some unique and some common feedback.

Table 1 Participant experience feedback from online survey (n=391) and interviews (n=14)

	Survey multiple choice	Survey open-ended questions	Qualitative interviews
Content
Easy to understand		ü	üI felt like the content was delivered in a way that made it very easy to understand [#14].
Relevant to practice, particularly/ case studies		ü	ü I think it was quite relevant. Especially when they brought in a case study that is very common in the general practice room [#12].
A good resource		ü	ü I think the main things were resources I could use to look up things in future if I had trouble. So, I saved them as bookmarks [#12].
Delivery
Flexible	ü (Q.5) Flexibility in learning	ü Flexible with time and pace	ü The fact that it was online, that I could do it in my own time, and that flexibility was great [#09].
Easy navigation / user friendly	ü (Q.6)	Mixed	Mixed The navigating the system was fine. It was easy [#04]. I did find a problem with it. When you had to move answers into boxes, if it accidentally went into the wrong box [#08].
Individualised learning	ü (Q.8)
Feedback /interactivity	ü (Q.11)	üLiked the interactivity e.g. quizzes & feedback	ü It was good that we were sort of asked to generate a response, to answer a question. I also enjoyed that they [the course] gave you a model answer [#14].
Engaging		ü	Mixed So, I thought it was good mix of media which made the presentation interesting and it kept my attention for longer than it may have otherwise [#07]. I found that it was a little bit difficult to engage with some of the modules [#3].

The responses to online questions on registrar satisfaction were mostly in the upper half of the rating scale, suggesting general satisfaction (Figure 2). Consistent with the qualitative data, the participants highly rated the ease of navigation (Q.6), the flexibility of the online delivery (Q.5) and the feedback (Q.11).

In contrast to the positive responses to Q.6, the open-ended question about what participants would like changed (Table S2) identified that many participants had problems with technical issues, and some thought the interface could have been more user friendly.

Data from the open-ended questions (Additional File 1, Tables S2 and S3) and interviews (Table 1) provided additional feedback on the course content: that it was easy to understand, provided relevant examples, it was relevant to practice, and it provided useful resources for future use.

RQ2. A. Impacts of course on attitudes, knowledge skills and clinical practice

Data from the survey and the interviews indicated that the participants thought the course positively increased their confidence, knowledge and skills of EBM. Data on changes to clinical practice were only available from the interview data collected at least three months after completion of the course. Interview data (presented below) suggested the course changed the participant’s clinical practice to better incorporate EBM.

Attitudes

Most participants said they were only interested in the application of research into practice. A few participants said that they were interested in doing or participating in research. Of those, few said that this was influenced by taking the course.

Participants mostly found themselves more confident in understanding, interpreting and appraising research evidence after completing the course.

I feel more confident, and I think that I would be able to if I needed to look through a study and make some comments [#08-female].

However, they acknowledged that they might not be able to conduct research, or there is room for improvements. A few stated that they did not find any change in their confidence.

I probably feel about the same as I did before. I don’t know that I feel any more or less confident [#10-female].

Knowledge

An increase in knowledge was identified in the interview and the online survey. Based on the survey findings, participants’ self-reported understanding of the topic substantially improved from the mean of 4.4 (out of 10) ± 0.1 (SD) before the course to 7.2 ± 0.1 after completing the course (n=320, p˂0.0001) which represents a very large effect size (d=1.6). Most of the participants believed that the course increased or refreshed their understanding of different research skills and awareness of how to incorporate research skills into their practice.

It’s a good idea I think to have an understanding of research when we’re going out into clinical practice [#05-female].

It [the course] makes me more aware to incorporate research into my practice [#07-male].

In addition, participants acknowledged that the course, led them to start questioning practice where clinicians follow others’ experience/opinion without thinking critically.

The way it’s [the course] influenced my management is just always being aware that just because it’s been done as usual practice doesn’t necessarily mean like it’s evidence-based and doesn’t mean that it’s necessarily proven to be effective. So, I don’t always rely on that advice [#01-male].

One participant highlighted an improvement in her understanding of the importance of contextual and environmental factors, including patient preferences in applying evidence.

It’s not just whether or not they’re sick, it’s also about how they approach health providers, and also how their general life impacts on whether or not they’re willing to engage or continue their treatment. So, I just keep it in the back of my mind that we need to be aware of the other socioeconomic and environmental factors which a lot of that qualitative research helps us consider as well [#14_female].

Skills

Some participants said they were now able to critique research evidence and interpret research studies.

It [the course] definitely taught me to be a bit more critical in my thinking of what I’m reading [#14-female].

I think it’s just helpful to …being able to interpret and synthesise how that can apply [research] to your clinical practice [#03-female].

Some participants reported that they learnt how to frame their clinical questions and find an evidence-based answer for it.

Instead of thinking that you don’t know something, and letting that overwhelm you, you become a little bit better at devising a clinical question and knowing where to answer is [#13_2-male].

Clinical practice

Some participants reported that the skills that they learnt had led to some changes in their practices. They used the skills learnt and research evidence to investigate an answer to their clinical questions particularly when they were dealing with uncertainty. They acknowledged that they used skills such as clinical appraisal, particularly the levels of evidence [#3,4,7,14] to interpret study findings, findings that drug representatives presented to them, and guidelines.

Looking at evidence and working out how reliable this is, and then using that to, you know, guide treatment or, you know, not needing to always rely on guidelines to make a better judgment, depending on the clinical situation after reviewing what evidence is available [#08_female].

… often I will have a lot of different clinical questions that I need to have answered. …and that’s what the course are really good for, for making that clinical question and then finding that answer, and knowing that it’s a trustworthy and reliable kind of answer that I actually apply to the patient [#13_2-Male].

A few participants indicated that the skills that they learnt in the course reaffirmed the importance of communication skills with patients. Their improved understanding of research findings and being able to elaborate the difference between high quality and low-quality evidence to patients reportedly improved their communication with patients.

It [the course] does help to clarify things in your own mind, which then explain it in a simple way to the Patient [#13_2-male].

I do bring it back to research and how just because one thing works for someone doesn’t mean that it works with you and just break it down that way, and also talking like, high-quality studies versus poor quality as evidence [03-Female].

RQ3: Mediators of impacts (Reported barriers and facilitators)

Responses from the qualitative interviews indicated that barriers and facilitators to practicing EBM related to the GP (GPs perceptions of EBM, comfort and priority); the work-place (time, the influence of supervisors, the impact of system and access to resources); and patients (treatment expectation being different from evidence).

GP factors

Most of the participants had positive attitudes towards EBM and acknowledged the value of EBM.

Obviously, we have to practice evidence-based medicine, so in order to do so, we need to be able to understand and interpret and incorporate research into our practice [#07-male].

Almost all participants reported they needed to seek information to inform their decisions on a daily basis. They reported that they tried to choose EBM resources that were recent and relevant to the Australian context and had confidence in the quality of the information.

All of them [the guidelines that I use] are sort of peer-reviewed and accepted by the wider community as factually correct [#11_2-male].

A few participants expressed a different opinion and described research evidence as neither relevant nor transferable into clinical practice.

I think they’re [research evidence] just answering questions that are quite different from the questions that we get in general practice [#06-female].

However, a few participants described an approach based upon trust in the credentials of the source of information:

I do feel if it’s [a research findings] published in a reputable source, I tend to leave it without thinking too critically… I trust my supervisor and feel that they are quite competent [#05-female].

A barrier that was described was, despite that the participants were willing to change, they felt more comfortable with what they have already learnt and get accustomed to, than using new evidence-based resources.

Often you’re introduced to something like an UpToDate [a resource for supporting clinical decision] quite early, so you get good at searching and using it, that you know what sort of services are on it [#13_2-male].

Work-place factors

Time was one of the main barriers identified for accessing and using research evidence in practice by almost all participants. The time-consuming nature of using research evidence was attributed to the way it was accessed, (for example, an initial need for login to the webpage of some organisations such as RACGP), the overwhelming amount of information identified by searches, and the time needed to critically appraise the findings.

If I were to go through Cochrane and look up a whole bunch of different articles which would take a lot longer [#05-female].

For almost all registrars, pre-appraised resources that are brief and ready to use, such as guidelines, were preferred over primary research evidence.

I would tend to use resources that are incorporated study findings into a summary like eTG [online Therapeutic Guidelines], I don’t read the specific articles and therefore analyse the data’ [#5-female].

Participants’ responses indicated that using research evidence might not be their priority; work and exams were specified as activities that they prioritised.

Because I do have exams coming up I haven’t been able to do – look into research papers [#11_2].

Participants reported thatsupervisors could berole models influencing GP registrars’ beliefs about evidence-based medicine by encouraging and guiding GP registrars to practice evidence-based medicine. Participants who said their supervisors encouraged them to use research evidence had a stronger belief than other participants about the applicability of research into practice.

Well, I think evidence-based research should inform good clinical practice, and it should always be the starting point for good management… Well, my supervisors have mostly been very evidence-based as well, so I’ve actually just learnt a lot from how they appraise studies [#03-female].

Some participants also indicated that some supervisors expect GP registrars to follow their advice and treatment approach without critically evaluating the relevance of the advice into the clinical situation.

[one of my supervisors is a] real old-school doctor, right, so they are less likely to change or read the literature or – this is the way we’ve always done it, so that’s the way they’ll always do it. So that’s the way they want me to do it as well. Which may not be – it’s not dangerous, but it may not be an optimal solution [#11_2_male].

One participant described an external system barrier to changing practice as an impediment to practicing EBM. In particular, he noted that the Pharmaceutical Benefits Scheme (PBS) system did not always allow the prescription of new medicine.

That’s a good skill [using research to inform the practice], but in most instances, it’s not applicable to general practice because we’re following these well-worn pathways in terms of the treatment and the investigation. An example of that might be, a study might come up with a fantastic new diabetes drug, well unless it’s on the PBS the vast majority of people don’t care because they would have to pay more for it. In terms of the incorporation of that drug into established treatment patterns, it goes through years of iterations before it’s widely accepted [#11-male].

Access to research papers was raised as a problem by some participants who indicated that having access could be costly unless they had free access through RACGP or other affiliations.

Patient factors

That’s mainly patient preference, so potentially they don’t want to go with the evidence-based therapy [#04-male].

Summary

Figure 3 illustrates the main themes identified to answer the study questions and their interactions. The interactions were interpreted from the qualitative data and whether the specified changes and influences were related to the course. In summary, participants had a positive experience from the course and stated that taking the course led them to improve their, confidence, knowledge, and skills.

Participants specified that they used the skills learnt in the course for interpreting evidence, investigating their clinical questions, and the course led them to have better communication with patients in their clinical practice. The factors that influenced the course's impact were specified to be related to GP, work-place, and patient factors.

This study evaluated GP registrars’ experience with an online training course that aimed to increase their knowledge of research methods and the application of critical appraisal. The course was a compulsory component of their vocational specialist training developed by content and educational-design experts. The data showed the registrars were generally positive towards the course and the concept of EBM. They stated that the course improved their confidence, knowledge, and skills and consequently impacted how they practice. Particularly, participants specified that they used the skills learnt in the course for investigating their clinical questions, interpreting evidence, and communicating with patients in clinical practice. The self-reported improved understanding of the topic increased with a very large effect size (d=1.6).

Participants in the online survey and qualitative interview reported that they expected a course to be interactive, relevant to practice and multi-media. For example, the course videos were appreciated by participants and acknowledged that it made the course engaging. This is aligned with research findings on EBM education (27–31) and principles of effective teaching (26) that support interactive and case-based learning. For example, Ilic et al. 2015 reported that participants preferred the YouTube videos and found it engaging compared to the lectures delivered for EBM teaching (36). Similarly, Cranney 2001 reported that GPs who participated in an educational program for evidence-based hypertension management preferred topics that were relevant to their daily practice (11).

Registrars identified multiple barriers and facilitators for implementing EBM. Of those, some were GP-related factors including GPs’ perception about EBM and comfort, some were work-place and system related, including the influence of supervisors, time and access to resources, and one was related to patient preferences. These factors identified in this study were similar to previous research influencing EBM practice (5,6,11,12). In this study only one participant found PBS as a system barrier for prescribing new evidence-based medicines; however, this has not been identified in earlier studies. Below we further focused and discussed the factors that might be addressed by educational interventions.

Given that GPs’ practice behaviours are established early (37) and remain stable over time, ensuring a supportive work environment for the practicing of EBM is likely to be important for the translation of learning from the course to practice. The influence of supervisors can be utilised to facilitate promoting EBM in practice in future educational interventions. In the current study, participants who were positive about the EBM stated that their supervisors had advised using research evidence in their practice. One of the ways that GPs registrars learn about prescribing is through teaching sessions and discussions with supervisors (38). Previous studies have also shown that GP registrars’ practices are likely to be influenced by the decisions of senior supervisors (18), and their practicing behaviours are likely to be similar to their supervisors (19). In addition, the relationship between supervisors and GP registrars can have a role in the process of practicing EBM. For example, in the current study, some registrars stated that supervisors wanted them to follow their advice without critically evaluating it. This could be a challenging situation for GP registrars because of the differential power and experience between registrars and supervisors. Similar to the current finding a study on barriers for practicing EBM among GP registrars also identified that registrars might lose their motivation to seek an answer for their clinical questions if the supervisor does not provide them sufficient autonomy (20).

Thus, further research might examine the effect of courses that incorporate discussion of what might GP registrars face in practice and how to deal with it. Research on interventions for GP supervisors to facilitate and encourage the practice of EBM by their registrars could also be beneficial. Given that discussion sessions between registrars and supervisors are an opportunity for registrars to learn, collaborative learning discussions can be utilised for development and translation of EBM skills into practice while creating a safe, equal and collaborative environment between registrars and supervisor (39). A previous study reported that this approach was useful for promoting EBM (39).

The effectiveness of future EBM training courses might be improved by providing strategies for responding to patient preferences that are at odds with research evidence. For example, a non-randomised controlled trial showed that GP registrars, who took part in online training on improving communication with patients, prescribed significantly fewer antibiotics compared to the control group (40).

Relevance of theory to study findings

As the objective of the course was to change behaviours of registrars, we considered our results in relation to two frameworks developed for behaviour change. We first compared our findings with theoretical domains proposed to identify the process specifically involved in changing the behaviour of the health professionals for practicing EBM (41). This framework identified specific domains that are directly relevant to the implementation of EBM training. Then, we applied an overarching framework -the behaviour change wheel (BCW)- which had been developed for improving the design and implementation of interventions for behaviour change in general (42).

There are 12 domains that were suggested for identifying the process of behaviour change of health professionals, proposed by Michie et al. (2005) after considering motivation theories (e.g. Social Cognitive Theory), action theories (e.g. Learning Theory and Organisation Theories (e.g. Goal Theory; Michie et al., 2005). Out of those nine domains were relevant to our findings. Table 2 illustrates our study findings across these nine domains. Three domains of ‘motivation and goals (Intention)’; ‘emotion (emotion)’; and ‘behavioural regulation’ were not identified in the current study. This could be because they were not directly relevant to the objectives of the course. Therefore, future studies/interventions could explore how healthcare professionals’ motivation and emotion affect practicing EBM. Further, there is room for research on the efficacy of promoting behaviour change techniques such as goal setting and self-monitoring for regulating healthcare professionals’ EBM behaviour in future interventions.

In addition, relating the behaviour change wheel (BCW) framework (42) with our findings suggests that the course was able to address two out of three essential conditions of ‘capability’, ‘opportunity’, and ‘motivation’ required for behaviour change. By providing two intervention functions (education and training), the course increased the GP registrars’ knowledge, understanding and skills of EBM. Consequently, it increased the registrar’s ‘capability’ (physical and psychological) to practice EBM. It also provided an ‘opportunity’ (physical) for registrars to practice what they had learnt by being implemented when learners were exposed to clinical cases (environmental restructuring). However, the conditions of ‘motivation (both automotive and reflective)’ and the ‘psychological opportunity’ were not identified as part of the course impact. According to the BCW framework, the two intervention functions of modelling and persuasion (using communication to induce positive or negative feelings) can be used to increased automatic and reflective motivation (42). Considering the influence of supervisors on registrars (19), they can serve as a role model for registrars to deliver modelling function. In addition, by reducing barriers and increasing means (e.g. behavioural support) the future course can increase opportunities and reflective motivation (42).

Table 2 Theoretical domains underpinning the study findings

Domains*	Study findings
Perceived impact	Barriers and enablers
Knowledge	Increase in knowledge of Finding/interpreting/critiquing research information	Technical issues
Skills/ability	Ability to Find/ interpret/critique/ use research information
Social/professional role and identity (Self-standards)	GPs self-standards ensured to choose the high-quality and EBM guideline
Beliefs about capabilities (self-confidence)	Increased confidence
Beliefs about consequences (Anticipated outcomes/ attitude)	Awareness of the importance of incorporating research into practice	Perception about EBM, considering supervisor recommended practice to be acceptable without questioning
Memory, attention and decision processes	Difficulty remembering to use research evidence
Environmental context and resources	Increased access to databases	Time, access to databases, system
Nature of the behaviours (Routine)	Past (existing) behaviour
Social influences (Norms)	Supervisors and accepted norms

*Michie et al. 2005(41)

Implications

The current study suggests that an engaging online course can be used as appropriate alternative training for EBM learning among GP registrars. Supervisors may have a considerable impact on registers’ motivation and their ability in translating their learnt EBM skills into practice by encouraging registrars and actively engaging in an equal, interactive discussion around EBM. Thus, future studies might get the benefits of simultaneous supervisor training and collaborative learning discussion method. In addition, future EBM learning courses may consider training registrars on how to respond to patients if their preferences are at odds with research evidence.

Limitations

There are several limitations to this study that should be acknowledged. The study only provided qualitative data on learner perspectives after finishing the course. Objective measures of knowledge, attitudes and behaviours were not collected from a control group that received no training and not collected before the intervention to measure changes over time. Consequently, the study is vulnerable to social desirability (telling us what we want to hear) and recall biases. Telephone interviews sample was homogeneous and lacking in a full spectrum of variation. In particular, only one student was trained overseas, and the experience of overseas-trained students could be different to those of locally trained students. Since the data collection ceased before the data satiation, we might have missed some information. However, there was good saturation in most of the themes.

This study showed that an online interactive and multi-media training that aimed to enable GP registrars to use research evidence in their practice changed registrars’ knowledge, confidence, skill and behaviour of EBM. The findings of this study can assist future educational interventions to be effective. We conclude that an interactive and engaging online course with relevant practical scenarios can be effective in teaching EBM. Further, this study highlights the importance of the supervisor’s role in GP registrars’ ability in translating the EBM skills learnt in to practice. Thus, studies might want to explore how educating supervisors can facilitate their role in supporting registrars to practice EMB. Incorporating training on how registrars can engage with patients regarding evidence might also be considered in future EBM course developments. In addition, considering theoretical frameworks for developing interventions suggest that investigating the role of motivation, emotion and behavioural change techniques on practicing EBM might be beneficial.

Ethics approval and consent to participate

Ethical approval was obtained from the Human Research Ethics Committee at UNSW Sydney (Approval number HC16483), and all methods were performed in accordance with the relevant guidelines and regulations.

Consent for publication

Not applicable

Availability of data and materials

The datasets generated and analysed during the current study are not publicly available because consent was not obtained from study participants for data to be made public but are available from the corresponding author on reasonable request subject to approval from the Human Research Ethics Committee at UNSW Sydney.

Competing interests

GP Synergy Ltd contracted UNSW, Sydney to develop the online course that is the subject of this paper. Specifically, CS, MH, CWMT and NZ were involved in the development of the modules. PM is employed by GP Synergy Ltd. No authors have a financial interest in the online course.

Funding

The project was conducted with funding from GP Synergy Ltd.

Authors' contributions

NZ, CS, MH and TL contributed to the conception and design of the project. HR-A collected qualitative data and conducted data analysis. CS, CT, MH, PM and TL contributed to the data analysis. CS provided study and research supervision. HR-A and CS drafted the manuscript and all authors contributed to the revision of it and providing feedback.

Acknowledgements

The research team would like to thank the GP registrars who participated in this study and shared their time and experience for the purpose of this research.

EBM, Evidence-based medicine; PBS, the Pharmaceutical Benefits Scheme

Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996 Jan 13;312(7023):71–2.
Albarqouni L, Hoffmann T, Straus S, Olsen NR, Young T, Ilic D, et al. Core Competencies in Evidence-Based Practice for Health Professionals Consensus Statement Based on a Systematic Review and Delphi Survey. Jama Netw Open [Internet]. 2018 Jun;1(2). Available from: ://WOS:000452640700002
Training curriculum [Internet]. Australian college of rural and remote medicine. [cited 2020 Oct 1]. Available from: https://www.acrrm.org.au/resources/training/curriculum
RACGP 2016 Curriculum [Internet]. RACGP. [cited 2020 Oct 1]. Available from: https://www.racgp.org.au/education/education-providers/curriculum/2016-curriculum
Young JM, Ward JE. Evidence-based medicine in general practice: beliefs and barriers among Australian GPs. J Eval Clin Pract. 2001 May;7(2):201–10.
Zwolsman S, te Pas E, Hooft L, Wieringa-de Waard M, van Dijk N. Barriers to GPs’ use of evidence-based medicine: a systematic review. Brit J Gen Pract [Internet]. 2012 Jul;62(600). Available from: ://WOS:000307745800025
Dallas A, van Driel M, Morgan S, Tapley A, Henderson K, Oldmeadow C, et al. Antibiotic prescribing for acute otitis media and acute sinusitis: a cross-sectional analysis of the ReCEnT study exploring the habits of early career doctors in family practice. Fam Pract. 2017 Apr 1;34(2):180–7.
Magin PJ, Morgan S, Tapley A, Davis JS, McArthur L, Henderson KM, et al. Reducing general practice trainees’ antibiotic prescribing for respiratory tract infections: an evaluation of a combined face-to-face workshop and online educational intervention. Educ Prim Care. 2016;27(2):98–105.
McColl A, Smith H, White P, Field J. General practitioners’ perceptions of the route to evidence based medicine: a questionnaire survey. BMJ. 1998;316(7128):361–5.
Te Pas E, van Dijk N, Bartelink MEL, Wieringa-De Waard M. Factors influencing the EBM behaviour of GP trainers: a mixed method study. Med Teach. 2013;35(3):e990-997.
Cranney M, Warren E, Barton S, Gardner K, Walley T. Why do GPs not implement evidence-based guidelines? A descriptive study. Fam Pract. 2001 Aug;18(4):359–63.
Taylor J, Wilkinson D, Blue IA, Dollard JT. Evidence-based rural general practice: barriers and solutions in South Australia. Rural Remote Health. 2005/05/07 ed. 2002 Jan;2(1):116.
van Dijk N, Hooft L, Wieringa-de Waard M. What are the barriers to residents’ practicing evidence-based medicine? A systematic review. Acad Med. 2010 Jul;85(7):1163–70.
Ma M, S L, A K, D AM, A AM, Zr M. Examining aptitude and barriers to evidence-based medicine among trainees at an ACGME-I accredited program [Internet]. Vol. 20, BMC medical education. BMC Med Educ; 2020 [cited 2021 Jan 13]. Available from: http://pubmed.ncbi.nlm.nih.gov/33167917/
Australian Commission on Safety and Quality in Health Care. Patient-centred Care - Improving quality and safety through partnerships with patients and consumers [Internet]. Sydney: ACSQHC; 2011 [cited 2020 Sep 3]. Available from: https://www.safetyandquality.gov.au/sites/default/files/migrated/PCC_Paper_August.pdf
Magin P, Morgan S, Wearne S, Tapley A, Henderson K, Oldmeadow C, et al. GP trainees’ in-consultation information-seeking: associations with human, paper and electronic sources. Fam Pract. 2015 Oct 1;32(5):525–32.
Kortekaas MF, Bartelink MLEL, Boelman L, Hoes AW, de Wit NJ. General practice trainees’ information searching strategies for clinical queries encountered in daily practice. Fam Pract. 2015 Oct;32(5):533–7.
Dallas A, van Driel M, van de Mortel T, Magin P. Antibiotic prescribing for the future: exploring the attitudes of trainees in general practice. Br J Gen Pract. 2014 Sep;64(626):e561-567.
Deckx L, Anthierens S, Magin PJ, Morgan S, McArthur L, Yardley L, et al. Focus on early-career GPs: qualitative evaluation of a multi-faceted educational intervention to improve antibiotic prescribing. Fam Pract. 2018 16;35(1):99–104.
Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Acad Med. 2005 Feb;80(2):176–82.
Ahmadi SF, Baradaran HR, Ahmadi E. Effectiveness of teaching evidence-based medicine to undergraduate medical students: A BEME systematic review. Med Teach. 2015 Jan;37(1):21–30.
Horsley T, Hyde C, Santesso N, Parkes J, Milne R, Stewart R. Teaching critical appraisal skills in healthcare settings. The Cochrane database of systematic reviews. 2011/11/11 ed. 2011 Nov 9;(11):CD001270.
Richmond H, Copsey B, Hall AM, Davies D, Lamb SE. A systematic review and meta-analysis of online versus alternative methods for training licensed health care professionals to deliver clinical interventions. BMC Med Educ. 2017 Nov 23;17(1):227.
Curran V, Lockyer J, Sargeant J, Fleet L. Evaluation of Learning Outcomes in Web-Based Continuing Medical Education: Academic Medicine. 2006 Oct;81(Suppl):S30–4.
Cook DA, Beckman TJ, Thomas KG, Thompson WG. Adapting web-based instruction to residents’ knowledge improves learning efficiency: a randomized controlled trial. J Gen Intern Med. 2008 Jul;23(7):985–90.
Boettcher JV. Ten Core Principles for Designing Effective Learning Environments: Insights from Brain Research and Pedagogical Theory. Innovate: Journal of Online Education [Internet]. 2007 Feb [cited 2020 Jul 15];3(3). Available from: https://www.learntechlib.org/p/171446/
Hatala R, Guyatt G. Evaluating the Teaching of Evidence-Based Medicine. JAMA. 2002 Sep 4;288(9):1110–2.
Hatala R. Is Evidence-based Medicine a Teachable Skill? Annals of Emergency Medicine. 1999 Aug 1;34(2):226–8.
Rolfe IE, Sanson-Fisher RW. Translating learning principles into practice: a new strategy for learning clinical skills. Med Educ. 2002 Apr;36(4):345–52.
McLean SF. Case-Based Learning and its Application in Medical and Health-Care Fields: A Review of Worldwide Literature. Journal of Medical Education and Curricular Development. 2016 Jan 1;3:JMECD.S20377.
Thistlethwaite JE, Davies D, Ekeocha S, Kidd JM, MacDougall C, Matthews P, et al. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23. Medical Teacher. 2012 Jun 1;34(6):e421–44.
Taylor R, Clarke L, Edwards D. Australian General Practice Training Program National report on the 2018 National Registrar Survey. [Internet]. Australian Council for Educational Research; 2018. Available from: https://research.acer.edu.au/higher_education/66
GP Synergy Research and Evaluation Strategic Plan [Internet]. Sydney: GP Synergy; 2017 [cited 2020 Sep 3]. Available from: https://research.gpsynergy.com.au/wp-content/uploads/2017/12/GP-Synergy-Research-Strategy_FINAL_External.pdf
Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006 Jan 1;3(2):77–101.
Liamputtong, Pranee. Rigour and ethics in qualitative research. In: Qualitative research methods. 3rd ed. South Melbourne, Vic.: Oxford University Press; 2009. p. 19–40.
Ilic D, Bin Nordin R, Glasziou P, Tilson JK, Villanueva E. A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine. Bmc Med Educ [Internet]. 2015 Mar 10;15. Available from: ://WOS:000351034300001
Björnsdóttir I, Kristinsson KG, Hansen EH. Diagnosing infections: a qualitative view on prescription decisions in general practice over time. Pharm World Sci. 2010 Dec;32(6):805–14.
Ajjawi R, Thistlethwaite JE, Aslani P, Cooling NB. What are the perceived learning needs of Australian general practice registrars for quality prescribing? BMC Med Educ. 2010 Dec;10(1):92.
Welink LS, de Groot E, Pype P, Van Roy K, van den Wittenboer ID, Bartelink M-LEL, et al. GP trainees’ perceptions on learning EBM using conversations in the workplace: a video-stimulated interview study. BMC Med Educ [Internet]. 2020 May 6 [cited 2021 Jan 13];20. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7201965/
Magin P, Tapley A, Morgan S, Davis JS, McElduff P, Yardley L, et al. Reducing early career general practitioners’ antibiotic prescribing for respiratory tract infections: a pragmatic prospective non-randomised controlled trial. Fam Pract. 2017/10/07 ed. 2018 Jan 16;35(1):53–60.
Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005 Feb;14(1):26–33.
Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011 Apr 23;6:42.

Competing interest reported. GP Synergy Ltd contracted UNSW, Sydney to develop the online course that is the subject of this paper. Specifically, CS, MH, CWMT and NZ were involved in the development of the modules. PM is employed by GP Synergy Ltd. No authors have a financial interest in the online course.

Additionalfile119Feb2021.docx

Download PDF

Editorial decision: Major revision
28 May, 2021
Reviews received at journal
23 May, 2021
Reviews received at journal
26 Apr, 2021
Reviewers agreed at journal
25 Apr, 2021
Reviewers agreed at journal
18 Apr, 2021
Reviewers invited by journal
07 Apr, 2021
Editor assigned by journal
22 Mar, 2021
Editor invited by journal
17 Mar, 2021
Submission checks completed at journal
17 Mar, 2021
First submitted to journal
19 Feb, 2021

You are reading this latest preprint version

Online training in evidence-based medicine and research methods for GP Registrars: a mixed-methods evaluation of engagement and impact

Status:

Version 1

Abstract

Background

Methods

Results

Conclusions

Figures

Introduction

The study objectives

Methods

Design

Recruitment and sample

Intervention: “the course”

Instruments and data collection

Data analysis

Ethics

Results

Participants

RQ1. Participant experience with the course

RQ2. A. Impacts of course on attitudes, knowledge skills and clinical practice

Attitudes

Knowledge

Skills

Clinical practice

RQ3: Mediators of impacts (Reported barriers and facilitators)

GP factors

Work-place factors

Because I do have exams coming up I haven’t been able to do – look into research papers [#11_2].

Patient factors

Summary

Discussion

Relevance of theory to study findings

Limitations

Conclusion

Declarations

Abbreviations

References

Additional Declarations

Supplementary Files

Status:

Version 1