Study design and clinical settings
We tested the QI set in the multicenter stepped-wedge, cluster-randomized controlled BRIDGE trial [22], which aimed to improve continuity and quality in rehabilitation for people with RMDs. The National Advisory Unit on Rehabilitation in Rheumatology recruited participating rehabilitation centers (n=8) in different regions of Norway. The centers started the trial simultaneously and acted as controls (delivering traditional rehabilitation programs) until an allocated point in time for each center to switch to the intervention phase (adding the new BRIDGE program to the traditional programs). Assessment of the responsiveness of the QI set was performed as a pre–post evaluation, before and after the addition of the new BRIDGE program.
Health professionals at the centers recruited patients at admission to rehabilitation. Patient-reported data were collected at admission and discharge from rehabilitation in secondary care and in the subsequent follow-up period at home (2, 7, and 12 months after admission). Eligible patients were aged ≥18 years and admitted to rehabilitation with one of the following diagnoses: inflammatory rheumatic diseases, systemic connective tissue diseases, osteoarthritis, osteoporosis, fibromyalgia or widespread pain, or non-specific low back, neck, or shoulder pain (persistent for more than 3 months). Because the electronic data collection and questionnaires were available only in Norwegian, patients needed to be proficient in Norwegian and to have a bank ID, internet connection, and a personal computer, tablet computer, or smartphone. Patients with fracture(s), cognitive impairment, or severe psychiatric disorder(s) were excluded. Eligible patients received verbal and written information about the study. Those who decided to participate provided written informed consent. The study was approved by the Norwegian Regional Committee for Medical Research Ethics (REK South-East, 2017/665).
The BRIDGE program
The main elements of the BRIDGE program are described in Table 1. At each center, the providers used a fidelity check list to monitor whether they delivered the program according to the BRIDGE protocol.
Data collection and measurements
At two time points, the head of each center completed the center-reported QI questionnaire in telephone-based interviews conducted by the central project coordinator (ALSS). The first interview was performed at the beginning of the study while the centers were still delivering traditional programs (T1). Using an interview guide based on the Scandinavian Team Arthritis Register-European Team Initiative for Care Research (STAR-ETIC) rehabilitation framework [23], the head of each center also gave detailed information about the content and organization of the rehabilitation program delivered at T1. The second interview took place 6–8 weeks after the addition of the BRIDGE program (T2).
Two months after the rehabilitation stay, all patients completed the patient-reported QI questionnaire. We collected patient-reported T1 data from patients participating in traditional rehabilitation programs (the T1-group) and T2 data from patients participating in the BRIDGE program (the T2-group). In this manner, we measured quality of rehabilitation services (at the institutional level) at T1 and T2 from the perspective of the users.
A QI set for the rehabilitation of people with RMDs
Providers completed a questionnaire addressing 19 structure indicators of quality. These indicators measured organizational aspects in which the rehabilitation occurs, e.g., whether written procedures, method descriptions, and/or checklists are currently available and part of the daily routine.
Patients responded to another questionnaire, comprising 14 indicators regarding process and outcome indicators of quality. Process indicators (n=11) measure factors related to giving and receiving care, in the form of actions and interactions between providers and patients in the actual clinical setting (20, 21). Outcome indicators (n=3) measure the effects of rehabilitation on defined outcomes, related to attainment of rehabilitation goals, improvements in function, and/or improvements in health-related quality of life [20,21]. Taken together, the main themes covered by the QI set are as follows: 1) patient participation in goal setting and the rehabilitation process; 2) follow-up plan and continuity across levels of care; and 3) assessment, outcomes, and time-points of evaluation. The QI set is presented in Table 2. The content of many structure indicators matches the content of process and/or outcome indicators, which allows for measuring quality in rehabilitation services from the system and user perspectives, respectively.
Because the elements in the BRIDGE program (Table 1) to a large degree mirror the items in the QI set (Table 2), we expected that the QI set would capture improved or maintained quality of rehabilitation between T1 and T2 (Figure 1). Maintained quality was favorable if the quality at T1 already was in line with the normative standards reflected in the quality indicators. If not, improved quality was favorable.
Response options and scoring alogrithm
Achievements (yes/no) of items in the QI set were measured using pass rates (PRs). Based on responses from the participant (provider or patient), calculations comprised single indicator PRs and total PRs. Single indicator PRs were calculated as the total number of participants who answered “yes” for a particular indicator divided by the total number of participants who answered “yes” or “no” for the same indicator. The scores were normalized to 100 to allow PRs to be reported as percentages.
Single indicator PRs range from 0% to 100% (100%=all eligible participants answered “yes” to this indicator). Total PRs represent the total of “yes” answers from a participant divided by eligible QI items (denominator) for the same participant. Eligible QI items in the center-reported questionnaire are always n=19. Eligible QI items in the patient-reported questionnaire are at least n=11 out of 14 but can vary. As an example: A patient who answers “yes” to question 1 (P1) goes to the additional question 2 (P2) (as seen in Table 2), resulting in n=11+1 for a denominator of 12. In the same way, an answer “yes” to question 7 (P7) makes questions 8a (P8a) and 8b (P8b) eligible, resulting in n=11+2, for a denominator of 13. Finally, “yes” answers to both questions 1 and 7 result in n=11+1+2, for the maximum denominator of 14. Total PRs also range from 0% to 100%, with 100% indicating the best quality in rehabilitation score, implying that the participant answered “yes” to all eligible items in the particular questionnaire.
The STAR-ETIC rehabilitation framework
Other measurements
We obtained demographic data about the patients at baseline. To assess the impact of data clustering from the multicenter design, we also used baseline data for the primary and secondary outcomes in the BRIDGE trial. The primary outcome was goal attainment, as measured by the Patient-Specific Functional Scale (PSFS) [25, 26]. Secondary outcomes were physical function, measured by the 30-second sit-to-stand test (30 secSTS) [27-29], and health-related quality of life (HRQoL), measured by the EuroQoL 5L-health-related quality of life (EQ5D-index and EQ5D-vas) [29, 30]). Norwegian versions of all instruments, translated following international guidelines, have been tested for psychometric properties with satisfactory results in RMD populations in rehabilitation settings in primary and secondary care [29].
On the PSFS (open-ended categories), patients report up to five activites that they currently find difficult to perform because of their health condition. Each activity is scored according to experienced performance on an 11-point scale (0–10, with 0 indicating “unable to perform”) [26,29]. In the EQ5D-index, patients report their level of perceived problems in five dimensions of health (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression; 5 levels, with 1 indicating no problems and 5 indicating extreme problems). In the EQ5D-vas, patients rate their current health state on a 100-mm visual analog scale (0–100, with 0 indicating “The worst health you can imagine” and 100 indicating “The best health you can imagine”) [29, 30]. In the performance-based test (30 secSTS), the patient, seated in a chair, rises to a full standing position and then sits down again. According to specific performance instructions, patients complete as many full stands as possible within 30 seconds [28, 29].
Responsiveness
Responsiveness has been defined by the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) panel as ‘‘the ability of an instrument to detect change over time in the construct to be measured’’ [31, p. 742]. In this study, we used a construct approach to examine responsiveness [33] because no gold standard is available. Based on current evidence, previous pilot testing [21], and the BRIDGE fidelity checklist, three of the authors (IK, GB, and ALSS) developed a priori hypotheses regarding the expected direction and magnitude of PR changes between T1 and T2. We discussed our hypotheses in a research group with nurses, patient research partners, and a physiotherapist. In accordance with de Vet [32], high responsiveness was indicated if at least 75% of the predefined hypotheses were confirmed.
The rationales for the hypotheses were based on results from the pilot study, other previous research, expert opinions, and fidelity checklist and guiding booklets available in the BRIDGE trial. The rationales are given in detail in Additional file 3. In short, we developed four hypotheses for median total PRs and 1–3 hypotheses for PR changes for each single indicator. We expected improved PRs for QIs that were addressed by the BRIDGE program: patient participation in 1) setting goals, 2) developing a written rehabilitation plan, 3) meeting(s) where goals and/or ongoing rehabilitation process were discussed, 4) consultation(s) about needs for the follow-up period, 5) developing a written follow-up plan, and 6) involvement of externals in planning follow-up. Concerning assessments and time-points of evaluation, we expected improved PRs for 1) use of reliable questionnaires/tests, 2) evaluation of goal attainment, function, and HRQoL at the start and end of the rehabilitation intervention in specialist care, and 3) 3–6 months after discharge (structure). We expected no change for QIs regarding initial bio-psycho-social assessment (process) and no change or little improvement for QIs regarding patient’s outcomes. Involving externals (i.e., next of kin or services in primary care) was expected as part of the follow-up plan, but invitations to meetings for next of kin or external services were not included in the BRIDGE program. Hence, we did not expect changes in QIs regarding invitation to meetings for next of kin or external services.
Data analysis
We used STATA IC v14 for statistical analysis. To compare the baseline characteristics of patients in the T1- and T2-groups, we used the independent samples t-test, Pearson’s Chi square test, and the Mann–Whitney U test. We set the significance level at 0.05. To assess the impact of clustering in each group, we calculated intraclass correlation coefficients (ICCs) for primary and secondary outcomes.
In testing hypotheses regarding responsiveness, we used descriptive statistics to examine the median PR values and change scores for total PRs and single indicator PRs, respectively.
Based on absolute changes, we used the following criteria for indicating the magnitude of changes: 1) 0%, no change; 2) 1.0%–12.5%, small change (change for 1/8 participating centers); 3) 12.6%–25%, moderate change (change for 2/8 participating centers); and 4) 25.1%–100%, considerable change (change for 3 or more participating centers). We used the same criteria for the magnitude of changes in patient-reported quality: 1.00%–12.5%, small change; 12.6%–25%, moderate change; and 25.1%–100%, considerable change.
Returned QI questionnaires were considered incomplete and not included in further analyses if more than 50% of the QI items had not received a “yes” or “no” response.