Poorly implemented interventions threaten participant and trial outcomes and undermine confidence in research findings. In intervention studies, it can be difficult to know whether interventions have been delivered as intended; that is, with fidelity (1, 2). However, despite the body of literature supporting the importance of fidelity, it is largely under-reported in studies of rehabilitation interventions (8–10). Without information regarding the extent to which an intervention has been delivered with fidelity, it is difficult to know whether the treatment effect outcomes are masked by poor implementation of the intervention (11). Fidelity data are necessary to interpret intervention outcomes (1). This is especially true of interventions with many interacting parts that are influenced by different contexts and factors, also called ‘complex’ interventions (12).
Complex interventions usually contain several ‘core components’ that are essential for the intervention to have an effect and to be considered as delivered with fidelity (5, 13). The higher the level of complexity and individual tailoring of the intervention and its components, the more difficult it may be to measure fidelity, (8, 14, 15), thus requiring a more sophisticated method of measurement to avoid drawing inappropriate conclusions about an intervention that might have been improperly implemented and making Type III errors (16). Measuring fidelity provides insight into which components of an intervention are essential for a positive participant outcome (17) by establishing what key components were or were not delivered in cases of improved outcomes (18).
Fidelity measurement is underpinned by theoretical concepts that emanate from behaviour change theories (1, 4, 19). Various frameworks have been developed to describe and define which aspects of intervention implementation should be considered and the methods to use when evaluating fidelity (3–5, 7). One such framework is the Conceptual Framework for Implementation Fidelity (CFIF) (3), which describes two key concepts to understanding implementation fidelity: 1) adherence (whether the recipient has received the intervention as intended) and 2) moderating factors (factors affecting faithful intervention implementation). Due to the comprehensiveness of this framework and its demonstrated usefulness in other complex intervention studies (6, 7), CFIF was used to define and describe fidelity in this study.
Some studies use quantitative data collection methods to measure elements of fidelity, such as fidelity checklists, that assess therapist adherence to core processes and determine which and core intervention components have been delivered (10, 20). Others use qualitative data collection methods, such as interviews, capturing acceptability or engagement with the intervention experienced by participants (15). These studies often do not include sufficient information regarding either the development of the fidelity measure used or the psychometric properties of the measure, which invites scepticism (10, 21–23). The lack of published studies detailing the development of fidelity measures emphasises the need for future research to make clear the processes used to assure good psychometric properties of the measure prior to its application.
There is a need for high-quality, psychometrically robust measures of fidelity, yet there is little agreement on how best to develop these measures (10, 23). Recent guidance suggests that to be considered high quality, the psychometric (e.g., reliability and validity) and implementation properties (practicality) of the measure should be reported (23). Evaluation of a measure’s psychometric properties can determine whether the scores consistently measure the intended constructs (8, 24). Practicality of a fidelity measure, such as ease of use and time taken to complete, is also valuable for researchers to report as these are factors that other researchers and clinicians consider when choosing a measure (8, 25).
Fidelity checklists are developed by using instructional information (i.e., intervention manuals), which is then distilled into a shortened list of intervention components and used to assess the presence of the components during delivery (23, 26). Checklists have the advantage of being simple and quick to administer by those without specialist training in the intervention itself, and in instances where study participants cannot be, or do not wish to be, recorded or interviewed (10). Assessment of fidelity through video or audio recordings of intervention delivery is currently considered the gold standard of assessment (50), but is resource intensive, especially in studies with many participants receiving intervention over an extended time period (23). Application of a fidelity checklist using retrospective review of intervention records might be a way to reduce resource use. Fidelity checklists have been generated in occupational therapy (8, 27), however, they are specific to the components of the various interventions they assess and inappropriate for use across studies of other interventions without adaptation (8, 17).
Vocational Rehabilitation (VR) is an example of a complex intervention that helps someone with a health problem return to or remain in work (28). VR involves helping people find work, helping those who are in work experiencing difficulties and supporting career progression in spite of illness or disability (29). VR is complex because it requires tailoring of the intervention to the individual receiving it, is sensitive to the behaviours of different stakeholders and can produce a variety of different outcomes (17). VR crosses organisational boundaries, involves interactions between multiple stakeholders, is highly individually tailored and requires behavioural change by the patient, their family and employer (30, 31). Stroke is an example of a particularly complex condition because it often occurs with multiple comorbidities and results in numerous, unpredictable biopsychosocial impacts (32). Delivering a particularly complex intervention (such as VR) in a complex patient group (such as stroke survivors) presents some challenges for intervention delivery and measurement of fidelity (such as tailoring and individualisation) to meet the specific needs of the recipients (33–35). A small number of studies describe VR for stroke survivors (36), but very few of these studies report whether VR was delivered with fidelity, which makes it difficult to draw firm conclusions about the effectiveness of VR after stroke (37) despite the existence of intervention non-specific fidelity measures (5, 38).
This study describes the development and testing of an intervention fidelity checklist for an early, stroke specialist vocational rehabilitation intervention (ESSVR) to support stroke survivors to return to work after stroke in the REurn To work After stroKE (RETAKE) trial (39) (ISRCTN12464275). ESSVR combines conventional VR with case management (see Fig. 1). It is delivered by a stroke-specialist occupational therapist (OT) trained to assess the impact of the stroke on the participant and their job; coordinate appropriate support from the UK National Health Service (NHS), employers and other stakeholders; negotiate workplace adjustments, monitor return to work and explore alternatives where current work is not feasible. A more detailed description of the intervention can be found elsewhere (40). ESSVR is delivered in four stages (early recovery, graded return to work, job retention and discharge), each comprising several core and desirable components.
[INSERT FIGURE 1]
Aims And Objectives
To develop and test a checklist for measuring implementation fidelity of ESSVR delivery in RETAKE.
Objectives:
(1) To identify and extract core ESSVR intervention components and generate guidance notes to assess the fidelity of their delivery within the RETAKE trial.
(2) To ratify the checklist components and guidance notes against expert opinion, supporting the measure’s content validity.
(3) To test the utility of the checklist for assessing fidelity of ESSVR delivery using retrospective observational review of stroke survivors’ intervention records.
(4) To assess interrater reliability in fidelity checklist completion.