The BCC recommendations were used as a guide to develop a comprehensive fidelity strategy for the RETURN trial, addressing the lessons learned from the conduct of the feasibility study. This article now sets out the strategy following the 5 domains (in bold) and goals (in bold italics) of the BCC framework (1), adapted by Borrelli (8).
Design
Explicitly identify and use a theoretical model as a basis for the intervention and ensure the intervention components and measures are reflective of underlying theory.
The theoretical underpinnings guiding the intervention have been detailed in a publication outlining the intervention's development process, which includes a comprehensive logic model (18). Briefly, the intervention draws from multiple theoretical frameworks, incorporating elements of Protection Motivation Theory (26) and Identity-based Motivation Theory (27).
During the feasibility study, it became evident that the conversational aspect of the intervention required a structured approach to enhance standardisation across intervention sessions. Accordingly, Motivational Interviewing (MI) ‘spirit’ (28) was introduced as a framework to provide structure to these conversations in the RETURN main trial, while bolstering the theoretical coherence of intervention deliveries.
Ensure consistent intervention dose and develop a monitoring plan to maintain consistency
Variations in intervention ‘dose’ were noted during the feasibility study, with session durations ranging from 10 to 37 minutes, Mean (SD) = 21 (8) minutes. Observations revealed this was primarily influenced by participant engagement and confidence levels of the interventionist.
Whilst an intervention duration target of around 15 minutes was set for pragmatic reasons as part of the intervention design goals for the feasibility study, for the RETURN main trial, a larger emphasis on dose standardisation was implemented through training. Additionally, to underscore the importance of regulating dose, specific guidance on the duration for each intervention component for the main trial was given:
Barrier discussion – 4 minutes
Motivation enhancement: video and discussion – 3 minutes
Knowledge enhancement (guided discussion using booklet materials) – 3 minutes
Setting Specific, Measurable, Achievable, Relevant, and Time-bound (SMART) goals and action plan – 4 minutes
Setting intentions at the session's conclusion – 1 minute
However, as a patient-centred approach is inherent in MI techniques where discussions are led by participants, variations in intervention dose were deemed an acceptable intervention adaptation in the RETURN main trial. This decision is supported by the understanding that participants facing multiple barriers may have longer ‘barrier discussions’ leading to variations in intervention duration. Monitoring of dose was achieved through audio-recordings, although no corrective measures were taken to standardise dose.
Participants also took intervention materials home, and accordingly, questions about additional engagement with the materials formed part of the RETURN main trial follow-ups. Likewise, metadata from the study website were reviewed to assess whether participants viewed intervention materials at home. This comprehensive approach to dose monitoring aims to enrich the interpretation of the RETURN main trial findings, and dose variations will be considered in the analysis of study outcomes (i.e. is there an optimum amount of ‘dose’ to elucidate behaviour change?).
Develop a plan for how adherence to the protocol will be monitored. Monitor both intervention delivery and assessment administration.
Adherence to the intervention protocol was identified as a concern during the feasibility study. Based on observations, only 5/11 (45%) intervention participants received the intended discussion eliciting perceived barriers to care from the participant. The following feasibility observation illustrates poor adherence to the prescribed approach:
DN02: “This is the pack; they have spoken to lots of people to make the pack. There are 6 barriers that people told to them. These are common and lots of people said them”.
Observation: DN02 was showing the ‘What Next Booklet’ to the participant but kept it in front of them so the participant was unable to read it. DN02 flashed the booklet and pointed to the barriers. Moving the booklet away again, they read the barriers out one by one.
DA0201: “So we have cost, time, I don’t think I have any problems, trust, embarrassment, anxiety”. The nurse turned the booklet over, and said “and there is also a plan, that is from psychological theory, and there are other materials”. All the while DA02 kept hold of the booklet.
Observation: DN02 then went back to the barrier page, showing the participant and asked: “Which of these do you relate to?” I felt this was quite a closed question. There was no conversation about what was stopping them from going. The participant was simply asked to choose which one from the list.
Observation 06: Site 02, DN02
To address this in the RETURN main trial, adherence monitoring considered the challenges of the research context. Indeed, findings from a recent scoping review of fidelity reporting in primary care dental settings (16) suggests the onus/burden of intervention protocol adherence and competency monitoring should sit with research teams. Therefore, in the RETURN main trial, dental nurses were asked to audio-record 100% of their intervention sessions, rather than alternative monitoring techniques such as asking them to complete checklists after each intervention delivery. The aim of this was to reduce the time and process burden on the dental nurses within the wider context of the RCT which, outside of the intervention delivery, has lengthy procedural requirements (e.g. consenting, randomising, data collection, data entry etc.). The use of audio-recordings for fidelity monitoring is considered the gold-standard (1), and whilst acknowledging that this method is researcher resource intensive, it was deemed the most appropriate method for use in the primary dental care context.. .
Adherence and competency during the RETURN main trial was then monitored by selecting at least one intervention recording per dental nurse each month which the research team scored using pre-determined criteria (the RETURN checklist, see Table 2).
Table 2
RETURN fidelity checklist
| RETURN fidelity checklist Dental Nurse Code: | Fully Implemented Score: 3 | Substantially Implemented Score: 2 | Partially Implemented Score: 1 | Not implemented Score: 0 |
| Overarching communication skills |
| Use of empathic listening statements | | | | |
| Use of relevant open questions | | | | |
| Use of non-judgemental language | | | | |
| Use of non-directive talk | | | | |
| Patient’s priorities, beliefs and challenges acknowledged | | | | |
| 1: Discuss barriers to regular dental attendance – raise awareness |
| Patient given space to tell their story (if they choose to do so) | | | | |
| Patient encouraged to come up with their own barriers without being led, or spoken for | | | | |
| Patient encouraged to decide on the one barrier they want to work on | | | | |
| 2: Increase Motivation |
| Patient is shown the video relevant to their selected barrier | | | | |
| Encouragement provided to the patient to reflect on the video, and how their own situation relates | | | | |
| 3: Increase knowledge |
| Patient guided to the booklet relevant to their selected barrier | | | | |
| Information (relevant to the barrier) provided to the patient | | | | |
| Statements communicated offering hope and assurances to the patients about their ability to overcome barriers | | | | |
| Emphasis placed on the benefits of regular dental attendance | | | | |
| 4: Setting SMART goals and action plans |
| SMART principles applied to goal and action plan | | | | |
| Patient guided to set their own goal and action plan tailored to their situation | | | | |
| Photographs of goal and action plan taken | | | | |
| 5: Increase Intention |
| Encouragement statements about what’s been achieved in the session | | | | |
| Encouragement provided to the patient to look at intervention materials at home | | | | |
| Feedback: |
| Strengths | |
| Areas for development | Unhelpful components present in session: ο Providing directive clinical advice ο Telling the patient what they should or should do / have done ο Setting goal / action plan for the patient ο Choosing the barrier for the patient |
The RETURN checklist comprised 6 essential intervention components: overarching communication skills (MI derived), barrier discussion, motivation enhancement through a video, knowledge enhancement through a barrier booklet, goal and action plan setting, intention setting. Each component comprised a combination of theoretical components designed to increase behaviour change capacity (i.e. encouraging the use of SMART principles for goal setting) and practical requirements (i.e. showing the video relevant to the selected barrier). The scoring system took the form of a Likert-scale: 0 = not implemented, 1 = partially implemented, 2 = substantially implemented, 3 = fully implemented, to give an indication of both adherence and competency.
Table 2
There are no guidelines to inform the optimum ‘level’ of fidelity that should be present in a BCI delivered within dental practices. However, Durlak and DuPre found outcomes were effective in educational interventions if they were delivered with 60%-80% fidelity (29), and a 90% threshold is frequently used in clinical interventions involving psychological therapies (30). Therefore, a cautious approach was adopted in the RETURN main trial and a threshold of 80% within each intervention component was used for a delivery session to be considered to have achieved high fidelity.
To provide guidance and to ensure consistency in intervention scoring, a scoring guidance manual was created (see Additional File 1). This was developed collaboratively by the RETURN researchers in the scoring team using an iterative approach to ensure that the descriptions contained within the manual were understood consistently across the team. The manual was both created and tested using a method whereby audio-recordings of the feasibility intervention sessions were scored independently, results compared, and discrepancies discussed until consensus was achieved (> 80% agreement rate).
The development of the fidelity checklist and the scoring guidance manual followed steps three to five as suggested by Walton and colleagues (31), with an iterative approach utilising feedback from the scoring team to refine the items and scoring guidance.
An example of the scoring guidance for ‘Overarching communication skills’ for the ‘use of empathic listening statements’ is illustrated below:
Patient’s priorities, beliefs and challenges acknowledged: patients should not be challenged on their beliefs, priorities or challenges experienced previously, even if they are in direct conflict with the principles of the delivery nurse. These should simply be acknowledged as an experience that occurred.
-
Score 0 if patient’s priorities/beliefs are challenged by the nurse e.g. Patient: “I couldn’t get a dentist because there weren’t any” Nurse “There was loads of NHS availability a year ago so that can’t be true”.
-
Score 1 if some attempt is made to acknowledge but the patient’s priorities/beliefs are also challenged e.g. Patient “I couldn’t get a dentist because there weren’t any” Nurse “It sounds like it was really difficult for you to get yourself into the dentist, but there were dentists available”.
-
Score 2 is patient’s priorities/beliefs and challenges are acknowledged during most of the session, but once or twice the nurse challenged the patients on these.
-
Score 3 if acknowledgments rather than challenges are present. Patient: “I couldn’t get a dentist” Nurse: “sounds like it was really tricky for you to get into a dentist in the past”.
Evaluation procedures to support scoring throughout the RETURN main trial also included the consistent use of the same scoring team and employing interrater reliability methods. Where an agreement rate of less than 60% was found between team members responsible for scoring throughout the course of the trial, additional scoring training took place, again using inter-rater reliability to determine agreement rates.
The RETURN checklist was designed as a multi-functional tool for the implementation of fidelity strategies. Its functions were to act as a standardised training aide, a method to set competency expectations, a means of leveraging feedback to interventionists, a means of monitoring protocol adherence and competency levels throughout the trial, and to assess the level of fidelity achieved in intervention deliveries at the end of the trial.
Develop a plan to record intervention protocol deviations and a method for providing timely feedback to interventionists
Several strategies were developed to document and address protocol deviations in the RETURN main trial:
-
A coaching culture was integrated into the training methodology to promote open communication and rapport between trainers and dental nurses. This aimed to facilitate an environment where protocol deviations would be more likely to be reported, and where feedback would be enacted. This took the form of regular, personalised, and constructive feedback designed to encourage confidence and build both communication and intervention skills.
-
Additionally, monthly, each nurse had at least one intervention audio-recording evaluated using the RETURN checklist with strengths and any areas for improvement noted. Checklists were provided to the dental nurses once completed. Where low scores were found, additional intervention sessions were scored, supplemented with a support site visit. Booster training was triggered where necessary through consistent low scores using the RETURN checklist.
-
The protocol deviation plan was clearly communicated to dental nurses at the outset of the RETURN main trial set-up phase. This transparent approach aimed to cultivate an environment where protocol deviations were viewed as opportunities for learning rather than punitive measures.
Develop a user-friendly scripted intervention manual to ensure consistency of delivery and adherence to active ingredients of the treatment
Learning from the feasibility study suggested that using scripted approaches to intervention delivery was unsuccessful, as was demonstrated in the following observation:
The nurse opened the pack and put it on the table. They read through the patient pack introduction information printed on the materials very quietly, not making eye contact with the patient as they did this. The patient was listening intently, leaning forward slightly to be able to hear what the nurse was saying.
I felt some of the meaning was lost during this explanation, as the nurse was so quiet and stilted, it was difficult to hear. The nurse came across as very unconfident and reliant on the written materials. This created no room for the patient discussion.
Observation 02: Site 02, DN02
For the RETURN trial therefore, a conscious move away from scripted materials was introduced, and instead, training intensity was increased. In addition, an easy-to-follow intervention crib sheet was developed (see Additional File 2), alongside a written intervention training manual, designed to support intervention delivery beyond training (see Additional File 3).
Plan for implementation setbacks
During the feasibility study, limited resources at sites resulted in the training of only one nurse per setting. As research activities were intended to integrate into nurses' regular duties within urgent dental care settings, this constraint contributed to recruitment delays, exacerbated by factors such as staff sickness or holiday leave.
To address these challenges in the RETURN main trial, three additional 'float' dental nurses were employed as part of the core research team to carry our research duties across sites, utilizing funds earmarked for reimbursing dental practices for staff time spent on research activities. Furthermore, efforts were made to train multiple dental nurses at each site, where feasible. These strategies were communicated to sites early during the setup phase.
Minimize contamination between conditions
Contamination was not found to be an issue within the feasibility study. Nonetheless, in the RETURN main trial, training was provided around the importance of allocation adherence. In addition, portable research activity flow charts detailing the specific actions to follow within each study arm were provided, supported by regular site visits from the research team.
Questions were included in the RETURN trial follow-up pertaining to contamination (i.e. control group question: ‘Did you receive any materials at your urgent care appointment to help you to find a dentist? If so, what did that look like?).
Training
Training was an identified as an area for improvement during the feasibility study.
‘Hiring’ dental nurses to deliver the RETURN intervention
Confidence was found to be a major contributing factor to intervention delivery success, detailed in the feasibility observation below:
I passed the booklet back to the nurse, and they started going through the booklet. They didn’t explain what the booklet was for. They read out the title on each page loudly, but the rest of the information on the pages was said very quietly and sound a little muddled. The walk through of the booklet didn’t flow, and it’s more like they were reading it to them themselves under their breath to familiarise themselves with the content.
DN02: (page 2) “For healthy teeth do I need to go?” “This is Megan, you can see about her story on the video”. Page 3 is skipped. Page 4 “Keeping on top of it” “It’s important to go all the time” Page 5 “This is a picture of a tooth that only the dentist could see, it shows the decay”.
It is very difficult to hear what DN02 was saying, and the overall feeling is someone who lacks confidence. I felt harsh making them deliver when clearly, they didn’t feel ready with any confidence.
Observation 05, Site 01: DN01
At the setup phase of the feasibility study, it was stipulated that effective delivery of the intervention would require experienced nurses with proficient communication skills. This expectation was based on the belief that such traits would facilitate the skills required to successfully deliver the intervention. However, implementation revealed challenges to this ideal. For example, the Principal Investigator (PI) at Site 02 reported using the study as an opportunity to enhance the communication skills and confidence levels of the nurse assigned to the research. This experience highlighted the existence of conflicting priorities when conducting research.
As we found that dental nurse attributes cannot be guaranteed, the RETURN main trial training included elements specifically designed to increase confidence and communication skills, including enhanced role-play and a coaching style training approach. In addition, training sessions were not fixed in length, and instead were based on individualised need. This was achievable as shadowing training occurred concurrently with participant recruitment, so did not hamper trial progress.
Standardise training
The BCC framework recommends training all interventionists together. In the primary dental care setting, this would require inviting dental teams to converge in a mutually convenient location, and taking staff members out of clinic was found to be problematic during the feasibility study. Instead, in the RETURN main trial, a model was used where site personnel were trained together. As this method could affect the standardisation of the training delivered, multiple strategies were designed to mitigate that risk:
-
Implementation of a ‘train the trainers’ training model led by a clinical psychologist
-
Using the same team of trainers throughout the trial
-
Using identical training materials for each site
-
Using the same role play tasks with all teams trained
-
Using a training manual and training videos
-
The development of a central website to house all training materials, as well as providing hard copies of all materials to all trainees
-
Use a training content checklist to ensure all training components were delivered to all dental teams (see Additional File 4).
Ensure dental nurse skill acquisition
Skill acquisition was not measured as part of the feasibility study. However, observations demonstrated variation in competency between dental nurses. Therefore, a plan was developed for the RETURN main trial to test skill acquisition during the different phases of training:
Training phase 1
Good Clinical Practice Training – A one-hour online module. Skill acquisition measured through an online quiz, with a pass mark of 80%.
Training phase 2
Intervention training – three hours, face to face delivery with a mixture of didactic learning, open discussions, and role plays. Skill acquisition measured through discussion and observations by a RETURN trainer through an intervention delivery skill acquisition checklist (see Additional File 5).
Training phase 3
On the job shadowing training – the amount depended on demonstration of competencies. Skill acquisition measured through in vitro observations using the RETURN checklist. Each interventionist needed to achieve a score of 80% within each intervention component in a single session to be signed off as competent to deliver the intervention independently. Scoring was conducted by the RETURN trainers and scoring decisions were supported by the guidance manual.
Minimise ‘drift’ in dental nurse skills
Skills drift was not monitored during the feasibility study. However, from feasibility observations, it was discovered that intervention skills needed to be practiced regularly to be maintained. Therefore, a strategy to reduce skills drift was developed for the RETURN main trial:
-
Frequent (at least one per month, per nurse) scoring and feedback of audio-recorded interventions using the scoring checklist, including elaborating strengths and areas for development
-
Triggered site visits to provide additional booster training and support in the event of low scoring (< 65% in any one component)
-
Triggered (by consistent low scores) or requested reflective practice sessions, wherein a selected audio-recording was discussed with the dental team at site, focusing on intervention elements that went well, and things that could be improved or done differently
-
Maintaining a collaborative coaching style approach to all feedback provision, booster training and reflective practice sessions to maintain relationships between the trainers and the dental nurses
Accommodate dental nurse differences
Stark differences between the skills and experience levels of the feasibility dental nurses were found. For example, DN02 was a trainee and lacked confidence with patient communication. DN01 had 12 years’ experience, demonstrated good communication skills and overall was more confident in their approach to the intervention.
This quotation from DN02 describes this:
Yeah. I don’t know it might be easy for other nurses but for my range of vocabulary to like GCSE, maybe some words I found difficult, and how it works, like the way it’s [training materials] worded was difficult. If it was more informal, like ‘What are we going to do?’ ‘We’re going to do this’. Like a chatty kind of presentation maybe
Interview with DN02
There were also differences in day-to-day responsibilities within their respective dental practices, with DN01 taking a more patient engaged role than DN02.
These contrasting quotations demonstrate this:
It’s very difficult, you know, especially for nurses because they do not have a lot of contact with patients. It’s only the dentist that takes over everything. So we do our own bit in surgery, cleaning, helping, but we don’t have conversations like that with patients.
Interview with DN02
I like talking to patients and I like the interaction and chatting with them and, you know, talking to different people as well and finding out their barriers. I think we seem a bit more human to them as well when we sit down and have a chat with them and we’re not just the scary people who work in the dentist
Interview with DN01
An additional challenge identified during the feasibility study was the need for training in the RETURN main trial to encompass multiple methods, accommodating a wide range of baseline skill levels. This was highlighted by the following observation on the first day of recruitment at site 02:
The nurse [DN02] told me that during the feasibility study training, they didn’t know what the word feasibility meant. They described that this word was in big letters on the very first training slide and all they could think about was wanting to Google what that word meant, so found it difficult to keep up with the rest of the training
Observation 01, Site 02: DN02
To maintain training standardisation whilst also acknowledging the challenge of variation between nurses likely be experienced in the main trial, an ‘on-the-job shadowing’ training element was employed.
Shadowing training involved a RETURN team member ‘chaperoning’ a dental nurse whilst they delivered interventions. Tailored support was provided alongside real-time verbal and written feedback. This training was not time limited. Training continued until the nurse both demonstrated competency through the scoring checklist and articulated to the trainers that they felt they has achieved a level of confidence sufficient to deliver the intervention independently. This style of ‘on-the-job’ shadowing training was selected for its ability to be highly individualised, and because it reflected the stye of training routinely undertaken by dental nurses in primary care.
Enhance buy-in from dental nurses
Enhancement of dental nurse buy-in was considered a priority for the upcoming RETURN trial. Within the dental practice setting, a practice owner often acts as the gatekeeper to research conduct. Those carrying out the research become involved later in the process, with vital opportunities to increase buy-in often missed. Accordingly, a series of dental nurse buy-in strategies were developed for implementation in the RETURN main trial:
-
Continuing Professional Development (CPD) accreditation for all training
-
Training components designed to explain the purpose of the research, paying particular attention to patient benefit
-
An early interactive information session including dental nurses, highlighting the opportunities presented by the trial for enhanced patient interaction and training
-
Inclusion of communication skills training targeted to dental nurses
-
Monthly newsletters aimed at dental nurses and wider practice staff, with the addition of real dental nurse stories about their involvement in the trial and a quiz and prize element
-
Engagement lunches for dental nurses as a reward for participation
-
Use of communication modes congruent with dental nurse preferences i.e. WhatsApp messages rather than emails
-
Regular site visits to increase self-efficacy and confidence with research activities
-
Dental nurse awards evening to celebrate trial achievements (i.e. best recruiter etc.)
Delivery
Use a scripted curriculum or treatment manual
Based on feasibility observations, scripts were not utilised in the REURN main trial. Instead, a selection of prompts were provided to the nurses to ensure the intervention’s essential components were delivered. These prompts took the form of the training manual (including intervention delivery cheat sheets), the intervention crib sheet, and videos demonstrating intervention delivery. Some components however, were ‘scripted’ within the intervention materials themselves, such as the goal and action planning section (see Additional File 6).
Assess non-specific effects through multiple methods and on an ongoing basis
Non-specific factors (such as empathy and components that lend themselves to the target communication style) were assessed as a stand-alone domain within the RETURN checklist. Nonspecific effects were also specifically discussed during shadowing training.
Ensure both adherence to the protocol and competency of intervention delivery
Adherence and competency of intervention deliveries were assessed through the application of the RETURN checklist throughout the trial. In addition, 100% of all available recordings were assessed at the end of the study to provide a comprehensive overview of the adherence and competency of intervention deliveries. A fidelity threshold of 80% in every domain per intervention delivery was applied when scoring the recordings.
Receipt
Ensure participants’ understanding of the intervention
In order enhance participants’ understanding of the intervention, the following steps were taken:
The RETURN intervention was designed to be engaging, specifically targeted to the trial population. An extensive patient and public involvement work stream fed into its design (full details have been published elsewhere (18)), with the aim of ensuring the materials were culturally relevant, containing congruent messages and images to the trial population. A design company and a professional illustrator were employed to embed these strategies.
To account for different learning styles, information was presented and repeated using multiple formats - verbal, written, pictorial and videography.
The intervention materials were written to a reading age of 8 years to ensure health literacy inclusivity.
Intervention delivery sessions were formatted as reciprocal conversations, and therefore by design, mutual understanding between the participant and dental nurse was embedded. During training and throughout the recruitment period, intervention deliveries were scored, and feedback provided to ensure that ‘reciprocity’ and patient understanding was embedded, with these criteria factored into the RETURN checklist.
Ensure participants’ ability to perform behavioural skills
The RETURN intervention seeks to target the behaviour of routine dental appointment visiting. To ensure participants’ ability to perform the behavioural skills required, the intervention was designed to be tailored, considering obstacles unique to individuals’ lives. The intervention culminates in a goal setting and action planning exercise, where participants think through their individual circumstances, and write out SMART (specific, measurable, achievable, relevant and time-bound) (32) gaols and plans to help them to overcome their barriers. In this way the target behavioural skills were articulated, discussed and broken down into small actions.
From the feasibility observations, this element of the intervention needed improvement, specifically around participant engagement.
The nurse put the booklet to one side, and then took the planning booklet from their knee. “We know that writing plans helps”. I felt this introduction didn’t really explain to the participant what the nurse was asking them to do – The nurse looked at me to help as they were getting their words muddled…The nurse devised the plan for the participant, rather than letting the participant make the plan for themselves. The patient set their goal themselves, but they did not put in much detail. They wrote down 3 words and didn’t discuss this with the nurse at all.
DN02, Observation 11
For the RETURN main trial, several strategies were implemented to improve how assessment of behavioural skills were conducted during the intervention delivery sessions:
Training included a dedicated component on how to facilitate goal setting and action planning, emphasising the importance of facilitating and not leading the task, and how to encourage participants to think through and articulate their own mechanisms.
Goal setting and action planning were included on the RETURN checklist, with timely feedback provided.
A follow-up text message was sent to participants a week post-intervention including the participants’ own wording from their goals and plans set within the intervention sessions to reinforce behavioural skills and build self-efficacy.
The 6-month follow-up telephone call to participants explored their comprehension of the intervention and how meaningful they found it to track receipt.
A component of the intervention conversation encouraged discussion around what was achieved during the intervention session. This was designed to improve participant receipt of the intervention by setting intentions. This element was also included in the RETURN checklist assessment.
Enactment
Participant performance of the intervention skills will be assessed in settings in which the intervention might be applied
Follow-up telephone calls in the RETURN main trial included a series of questions about whether and how the intervention materials and associated intervention skills had been used since leaving the urgent care dental setting. Questions focused on which parts of the intervention had been used, whether the intervention skills had been enacted (i.e. phoning for a dental appointment, exploring which dental practice they may like to contact, attending a dental appointment) and how the intervention supported any actions taken to attend a routine dental appointment.
Additionally, enactment strategies were embedded within the intervention materials themselves. Some materials are labelled ‘to look at at home’, providing encouragement and support in locating a dentist, making an appointment and thereafter attending an appointment – the behaviours targeted by the intervention.
The full RETURN fidelity strategy is summarised in Table 3.
Table 3
RETURN Fidelity Strategy Summary
| BCC Recommendation | RETURN Strategy | Method (where applicable) |
| Design |
| Explicitly identify and use a theoretical model as a basis for the intervention and ensure the intervention components and measures are reflective of underlying theory | State theoretical underpinnings of the intervention | Published paper outlining intervention development with logic model (18) |
| Ensure consistent intervention dose and develop a monitoring plan to maintain consistency | Specify target dose (15 minutes), provide timings breakdown of intervention components | Incorporate dose awareness into training. Dose is an allowable adaptation for RETURN, but dose will be explored in results |
| Develop a plan for how adherence to the protocol will be monitored. Monitor both intervention delivery and assessment administration | Recordings of intervention delivery sessions of each interventionist were assessed throughout the trial. 80% fidelity threshold within each intervention component required. A fidelity scoring guidance was developed | Audio-recordings of intervention delivery sessions. Iterative development of the scoring guidance with interrater reliability measures to ensure consistent assessment administration |
| Develop a plan to record intervention protocol deviations and a method for providing timely feedback to interventionists | Use a coaching style with training. Provide monthly feedback on sessions in a transparent way, using the RETURN checklist, supplemented with booster training and site support visits | Audio-recordings of intervention delivery sessions |
| Develop a user-friendly scripted intervention manual to ensure consistency of delivery and adherence to active ingredients of the treatment | Instead of scripts, the use of easy to digest crib sheets and supporting training materials designed to be used in intervention sessions | |
| Plan for implementation setbacks | Hire float dental nurses to carry out research tasks as needed, as well as to support nurses at sites. In addition, a plan to train multiple nurses at each site | |
| Minimize contamination between conditions | Training and the development of crib sheets that set out research activities for each arm. Questions were included in the trial follow-up to explore contamination with participants | Incorporate contamination awareness into training plans. Participant self-report at follow-up |
| Training |
| ‘Hiring’ dental nurses to deliver the RETURN intervention | Training to incorporate ‘soft skills’ necessary to foster intervention delivery skills and increase confidence. The use of a coaching style throughout the length of the trial | Observations |
| Standardise training | Use of ‘train the trainers’ model with the same team of trainers throughout the trial, use of identical training materials for all sites, the development of a central website to house all training materials, in addition to the provision of hard copies for all dental nurses | |
| Ensure dental nurse skill acquisition | The use of tests and pass scores for all training modules, utilising different methods (written, oral and action based) | |
| Minimise ‘drift’ in dental nurse skills | Assessment of intervention sessions for all nurses throughout the trial, triggered booster training, triggered or requested reflective practice sessions, coaching style maintained | Audio-recordings of intervention delivery sessions over time |
| Accommodate dental nurse differences | On-the-job shadowing training element, which provides real-time feedback and offers tailored training to suit individual needs | Observations |
| Enhance buy-in from dental nurses | CPD hours, training to include information about potential patient benefit, inclusion of soft skills development in dental nurse training, including dental nurses in early discussions with dental sites, regular engaging newsletters aimed at dental nurses, engagement lunches, regular site visits, use of communication modes congruent with dental nurse preferences, awards evening to reward achievements | |
| Delivery |
| Use a scripted curriculum or treatment manual | Instead of scripts, provision of a training manual with cheat sheets, cribs sheets and videos. In addition, some of the intervention materials were used as a ‘script’ to enhance standardisation | |
| Assess nonspecific effects through multiple methods and on an ongoing basis | Scored using the RETURN checklist throughout the trial, with feedback provided where necessary | Observations and audio-recordings |
| Ensure both adherence to the protocol and competency of intervention delivery | Scored using the RETURN checklist, with 100% scored at the end of the trial to provide an overview of adherence and competency in the trial | Observations and audio-recordings |
| Receipt |
| Ensure participants’ understanding of the intervention | The intervention was developed with the target population at the heart of its design, ensuring the intervention was engaging, culturally relevant and aesthetically pleasing. The intervention was presented in multiple formats to engage different learning styles. The intervention materials were developed to an appropriate reading age to facilitate health literacy inclusivity. Reciprocity is embedded within the intervention delivery, and enhancement of patient understanding embedded within the structure of the intervention session | Observations, audio-recordings |
| Ensure participants’ ability to perform behavioural skills | Goal and action plan facilitation is a component of the intervention where behavioural skills will be articulated. Training focuses on how to facilitate tailored goals and action plan setting. A follow-up text message forms part of the intervention to encourage behavioural skills. Questions around behaviour skills forms part of the trial follow-up with participants. Intention setting is included as part of the intervention where behavioural skills will be reaffirmed. Audio-recordings were scored against the RETURN checklist | Observations, audio-recordings, participant self-report |
| Enactment |
| Participant performance of the intervention skills will be assessed in settings in which the intervention might be applied | Questions were included in the 6-month telephone follow-up around enactment of the intervention skills | Participant self-report |
Table 3