Process-related findings A total of 215 unique health care providers, including 73 prescribers and 142 other clinic staff from the enrolled 4 family medicine and 5 internal medicine clinics completed at least one component of the QI intervention (QI participants; Table 1). Among the QI participants, 48.4% completed half or more of the intervention components; 44.7% completed at least 4 of the 6 in-person practice facilitation sessions; 31.2% completed the opioid prescribing and 23.2% completed the shared decision making online modules (Table 1). The intervention participation was voluntary, and not all clinic health care providers participated; although data on the total clinic staff were not collected or available, it was estimated that fewer than 50% of each clinic’s staff received the intervention.
Table 1. Completion of the intervention components among the intervention clinics’ staff.
Position
|
Completed ≥1 intervention component
|
Completed ≥50% of the intervention components*
|
Completed Academic Detailing
|
Completed Online Opioid Module
|
Completed Online Shared Decision Making Module
|
Completed ≥1 Practice Facilitation Session
|
Completed ≥4 Practice Facilitation Sessions
|
WAVE 1 QI INTERVENTION CLINICS
|
Clinic 1: Internal Medicine (urban, residency clinic)
|
MD, DO, #
|
7
|
5
|
6
|
4
|
3
|
6
|
2
|
NP, PA, #
|
3
|
0
|
1
|
1
|
0
|
3
|
1
|
RN, MA, CMA, LPN, #
|
17
|
9
|
1
|
10
|
4
|
16
|
10
|
Other staff, #
|
7
|
1
|
0
|
1
|
0
|
1
|
1
|
Total, #
|
34
|
15
|
8
|
16
|
7
|
26
|
14
|
Clinic 2: Internal Medicine (urban, non-residency clinic)
|
MD, DO, #
|
8
|
5
|
7
|
5
|
4
|
1
|
1
|
NP, PA, #
|
0
|
0
|
0
|
0
|
0
|
0
|
0
|
RN, MA, CMA, LPN, #
|
24
|
14
|
23
|
11
|
9
|
6
|
6
|
Other staff, #
|
8
|
2
|
4
|
2
|
2
|
2
|
2
|
Total, #
|
40
|
21
|
34
|
18
|
15
|
9
|
9
|
Clinic 3: Family Medicine (rural; residency clinic)
|
MD, DO, #
|
8
|
3
|
6
|
3
|
3
|
5
|
2
|
NP, PA, #
|
1
|
1
|
1
|
1
|
1
|
1
|
1
|
RN, MA, CMA, LPN, #
|
9
|
8
|
8
|
2
|
2
|
8
|
8
|
Other staff, #
|
4
|
1
|
3
|
0
|
0
|
5
|
2
|
Total, #
|
22
|
13
|
18
|
6
|
6
|
19
|
13
|
WAVE 2 QI INTERVENTION CLINICS
|
Clinic 4: Family Medicine (urban; residency clinic)
|
MD, DO, #
|
4
|
3
|
3
|
2
|
1
|
4
|
3
|
NP, PA, #
|
1
|
0
|
0
|
0
|
0
|
1
|
0
|
RN, MA, CMA, LPN, #
|
6
|
3
|
3
|
4
|
4
|
5
|
5
|
Other staff, #
|
3
|
2
|
3
|
1
|
1
|
3
|
2
|
Total, #
|
14
|
9
|
9
|
7
|
6
|
13
|
10
|
Clinic 5: Family Medicine (urban; residency clinic)
|
MD, DO, #
|
11
|
5
|
9
|
4
|
2
|
4
|
4
|
NP, PA, #
|
4
|
1
|
3
|
0
|
0
|
2
|
1
|
RN, MA, CMA, LPN, #
|
11
|
3
|
10
|
3
|
3
|
3
|
2
|
Other staff, #
|
8
|
0
|
0
|
0
|
0
|
5
|
4
|
Total, #
|
34
|
9
|
22
|
7
|
5
|
14
|
11
|
Clinic 6: Internal Medicine (urban; residency clinic)
|
MD, DO, #
|
7
|
1
|
7
|
0
|
0
|
6
|
2
|
NP, PA, #
|
3
|
1
|
3
|
1
|
0
|
3
|
1
|
RN, MA, CMA, LPN, #
|
14
|
5
|
12
|
2
|
1
|
14
|
5
|
Other staff, #
|
3
|
0
|
0
|
0
|
0
|
2
|
0
|
Total, #
|
27
|
7
|
22
|
3
|
1
|
25
|
8
|
WAVE 3 QI INTERVENTION CLINICS
|
Clinic 7: Family Medicine (suburban; non-residency clinic)
|
MD, DO, #
|
3
|
3
|
2
|
1
|
1
|
3
|
3
|
NP, PA, #
|
1
|
1
|
1
|
1
|
1
|
1
|
0
|
RN, MA, CMA, LPN, #
|
7
|
5
|
6
|
2
|
2
|
6
|
5
|
Other staff, #
|
2
|
0
|
0
|
0
|
0
|
2
|
1
|
Total, #
|
13
|
9
|
9
|
4
|
4
|
12
|
9
|
Clinic 8: Internal Medicine (urban; residency clinic)
|
MD, DO, #
|
4
|
1
|
3
|
1
|
1
|
2
|
1
|
NP, PA, #
|
2
|
2
|
2
|
1
|
1
|
1
|
1
|
RN, MA, CMA, LPN, #
|
6
|
4
|
5
|
0
|
0
|
6
|
4
|
Other staff, #
|
3
|
3
|
2
|
1
|
0
|
3
|
3
|
Total, #
|
15
|
10
|
12
|
3
|
2
|
12
|
9
|
Clinic 9: Internal Medicine (urban; residency clinic)
|
MD, DO, #
|
4
|
4
|
4
|
2
|
1
|
4
|
3
|
NP, PA, #
|
2
|
1
|
2
|
0
|
1
|
1
|
0
|
RN, MA, CMA, LPN, #
|
10
|
7
|
9
|
1
|
2
|
8
|
7
|
Other staff, #
|
0
|
0
|
0
|
0
|
0
|
0
|
0
|
Total, #
|
16
|
12
|
15
|
3
|
4
|
13
|
10
|
ALL QI INTERVENTION CLINICS
|
MD, DO, #
|
56
|
30
|
47
|
22
|
16
|
34
|
22
|
NP, PA, #
|
17
|
7
|
13
|
5
|
4
|
12
|
6
|
RN, MA, CMA, LPN, #
|
104
|
58
|
77
|
35
|
27
|
68
|
53
|
Other staff, #
|
38
|
9
|
12
|
5
|
3
|
23
|
15
|
Total, #
|
215
|
104
|
149
|
67
|
50
|
137
|
96
|
Legend
* 50% completion applies to clinicians/staff who completed two of the four intervention components: Academic Detailing, Online Opioid Module, Online Shared Decision Making Module, Practice Facilitation (4 of 6 sessions)
CMA: Certified Medical Assistant; DO: Doctor of Osteopathic Medicine; MA: Medical Assistant; MD: Doctor of Medicine; PA: Physician Assistant; NP: Nurse Practitioner; RN: Registered Nurse
Other process-related findings are detailed in Additional File 4. Briefly, pre-intervention, the clinicians/staff identified responsible opioid prescribing practices, shared decision making, and the management of patients with chronic pain as areas of educational need. Based on the post-intervention evaluation, this need was met through the QI intervention’s components. Overall, the intervention participants identified learning through the project about the conduct of a QI process and its impact assessment, and working better as a team as important outcomes of the project and useful skills that are “transferrable” toward other initiatives.
Baseline characteristics of the target patient population Across the 26 evaluated primary care clinics, 3,148 target patients (58.1% women, mean age 53.3 (SD 13.8) years) were identified, with 1,431 in the QI and 1,717 in the comparison clinics. These patients comprised 1.9% of the QI and 2.0% of the comparison clinics’ adult patient panels. A comparison of weighted, clinic-level baseline characteristics showed that the target patients in the QI and comparison clinics (Table 2) did not differ in a statistically significant way in relatively high daily MED doses and the rates of opioid-benzodiazepine co-prescribing. They also did not differ in their overall low rates of “current” treatment agreements, urine drug testing, completed opioid risk assessment, and documented PDMP check. However, the QI clinics, relative to the comparison clinics, had higher rates of completed depression screening (8.1% (SD 10.4) vs. 1.1% (SD 1.3), p=0.019) and of patients prescribed MED ≥90 mg/day (23.0% (SD 8.8) vs. 15.5% (SD 7.3), p=0.038). No statistically significant differences in baseline characteristics were noted in a subgroup of target patients treated with MED ≥90 mg/day in the QI (N=359) and comparison (N=283) clinics.
Table 2: Baseline characteristics of the target patient population.
Characteristics
|
Intervention clinics at baseline
(N=1,431)
|
Comparison clinics at baseline
(N=1,717)
|
p value*
|
Women, % (SD)
|
57.7 (7.1)
|
60.7 (8.9)
|
0.400
|
Mean age, years, mean (SD)
|
53.6 (3.2)
|
53.8 (3.5)
|
0.864
|
Completed treatment agreement (past 12 months), % (SD)
|
24.8 (13.8)
|
29.2 (18.3)
|
0.531
|
Completed urine drug testing (past 12 months), % (SD)
|
24.7 (11.8)
|
31.3 (16.2)
|
0.285
|
Completed depression screening (past 12 months), % (SD)
|
8.1 (10.4)
|
1.1 (1.3)
|
0.019
|
Completed opioid misuse risk assessment, % (SD)
|
0.2 (0.4)
|
0.7 (1.8)
|
0.478
|
Documented PDMP Check (past 12 months), % (SD)
|
0.0 (0.1)
|
0.3 (1.2)
|
0.503
|
Co-prescribed benzodiazepines in at least 1 out of 3 past months, % (SD)
|
19.9 (4.3)
|
24.7 (7.4)
|
0.089
|
Percentage of the adult clinic population, % (SD)
|
2.0 (0.9)
|
2.1 (0.6)
|
0.906
|
MED (past 90 days), mg/day, mean (SD)
|
75.7 (29.7)
|
55.9 (19.4)
|
0.063
|
MED ≥ 90 mg/day (past 90 days), % (SD)
|
23.0 (8.8)
|
15.5 (7.3)
|
0.038
|
Legend
Population of adult patients with opioid-treated chronic non-cancer pain in the 9 intervention and 17 comparison primary care clinics: characteristics at baseline (January 2016) based on the equally-weighted clinic averages.
MED: Morphine-equivalent Dose; PDMP: Prescription Drug Monitoring Program; SD: standard deviation
* p value was determined using the two-sample means test comparing clinic-level values for intervention versus comparison clinic groups
Primary Outcome Analysis: Mixed-Effects Regression Analysis Regression analysis of monthly outcomes by clinic and prescriber recognized variations in the timing of the QI intervention from clinic to clinic within the QI clinic group. This analysis did not reveal statistically significant changes in outcomes in the stepped-wedge pre-post analyses, which contrasted the pre-intervention period with the combined intervention and post-intervention periods. Augmentation with data from the comparison clinics did not impact these findings. See Additional File 3 for detailed results.
However, when the evaluation specifically focused on the intervention period, separating it from the post-intervention period, several statistically significant changes were noted in the QI clinics, after accounting for the trends in the comparison clinics (see Additional File 3). The completion rate of new treatment agreements (incidence rate) increased in the QI clinics during the intervention months both in the overall target population (by 9.4%; p=0.023; 95%CI=[0.028,0.159]) and in the subgroup of patients treated with MED ≥90 mg/day (by 15.9%; p=0.044, 95%CI=[0.029,0.289]); these differences were not sustained post-intervention in either group. In addition, in the QI clinics, among the overall target population and in the high-dose subgroup, MED decreased post-intervention (by -2.75 and -6.50 mg/day, respectively), but not during the intervention period (0.80 and 12.43 mg/day, respectively).
Secondary outcomes: means tests Both the QI and comparison clinics improved on all outcomes among the target population (Table 3), except the prevalence of opioid-benzodiazepine co-prescribing, which improved in the comparison clinics only (p=0.006). When comparing the change in outcomes between the QI and comparison clinics, no statistically significant differences were noted. However, Cohen d effect sizes favored the QI clinics, except for opioid-benzodiazepine co-prescribing (a small ES in the intervention and a moderate ES in the comparison clinics).
Table 3. Target Patient Population: Change in Outcomes.
|
Change in Intervention clinics
(N=1,255 at exit)
|
Change in Comparison clinics
(N=1,632 at exit)
|
Change in Intervention versus Comparison clinics
|
Characteristics
|
Change from baseline
|
p value†
|
Cohen’s d
|
Change from baseline
|
p value†
|
Cohen’s d
|
Difference
|
p value‡
|
Cohen’s d
|
Completed treatment agreement (past 12 months), % (SE),
95% CI (LB,UB)
|
32.0 (6.7)
(18.8, 45.2)
|
<0.001
|
2.1
|
29.2 (6.7)
(16.1, 42.4)
|
<0.001
|
1.5
|
2.8 (9.6)
(-16.1, 21.6)
|
0.816
|
0.1
|
Completed urine drug testing (past 12 months), % (SE),
95% CI (LB,UB)
|
33.4 (7.0)
(19.8, 47.1)
|
<0.001
|
2.6
|
25.7 (5.3)
(15.3, 36.1)
|
<0.001
|
1.7
|
7.7 (8.7)
(-9.3, 24.7)
|
0.277
|
0.4
|
Completed depression screening (past 12 months), % (SE),
95% CI (LB,UB)
|
49.2 (6.1)
(37.2, 61.1)
|
<0.001
|
3.4
|
55.4 (6.9)
(41.8, 69.0)
|
<0.001
|
3.3
|
6.2 (9.6)
(-25.1, 12.6)
|
0.526
|
0.3
|
Completed opioid misuse risk assessment, % (SE),
95% CI (LB,UB)
|
5.4 (3.0)
(-0.5, 11.3)
|
0.070
|
0.9
|
10.1 (5.8)
(-1.4, 21.5)
|
0.050
|
0.7
|
-4.7 (7.4)
(-19.2, 9.9)
|
0.417
|
0.3
|
Documented PDMP* Check (past 12 months), % (SE),
95% CI (LB,UB)
|
50.7 (6.3)
(38.3, 63.1)
|
<0.001
|
3.8
|
39.9 (7.8)
(24.6, 55.2)
|
<0.001
|
2.0
|
10.8 (10.8)
(-10.4, 32.0)
|
0.301
|
0.4
|
Co-prescribed benzodiazepines in at least 1 out of 3 past months, % (SE),
95% CI (LB,UB)
|
-1.2 (1.6)
(-4.3, 1.9)
|
0.461
|
0.3
|
-5.4 (1.8)
(-9.0, -1.8)
|
0.006
|
0.7
|
4.3 (2.7)
(-0.9, 9.5)
|
0.143
|
0.7
|
Percentage of the adult clinic population, % (SE),
95% CI (LB,UB)
|
-0.3 (0.2)
(-0.6, 0.0)
|
0.042
|
0.4
|
-0.4 (0.1)
(-0.6, -0.1)
|
0.056
|
0.5
|
0.0 (0.2)
(-0.3, 0.4)
|
0.547
|
0.1
|
MED (past 90 days), mg/day, mean (SE),
95% CI (LB,UB)
|
-11.6 (2.4)
(-16.4, -6.9)
|
<0.001
|
0.4
|
-9.9 (3.0)
(-15.8, -4.1)
|
0.0001
|
0.6
|
-1.7 (4.0)
(-9.6, 6.2)
|
0.937
|
0.2
|
MED ≥ 90 mg/day (past 90 days), % (SE),
95% CI (LB,UB)
|
-3.5 (1.1)
(-5.8, -1.3)
|
0.002
|
0.4
|
-1.6 (0.7)
(-3.0, -0.1)
|
0.004
|
0.2
|
-2.0 (1.3)
(-4.5, 0.6)
|
0.270
|
0.7
|
Legend
Population of adult patients with opioid-treated chronic non-cancer pain in the 9 intervention and 17 comparison primary care clinics; characteristics at exit (December 2017) and their patient-weighted change from baseline.
CI (LB,UB): Confidence Interval (Lower Bound, Upper Bound); MED: Morphine-equivalent Dose; PDMP: Prescription Drug Monitoring Program; SE: standard error
* Exit data for the PDMP outcome was collected in March 2017; starting in April 2017, the PDMP check documentation rose to approximately 100% across all clinics due to the changes in documentation requirements.
† p value was determined using the single sample mean test comparing clinic-level changes in outcomes from baseline to exit
‡ p value was determined using the two-sample means test comparing clinic-level changes in outcomes from baseline to exit for intervention versus comparison clinic groups
Among the subgroup of target patients prescribed MED ≥90 mg/day (Table 4), all outcomes tended to improve in both the QI and comparison clinics. Comparing the change in outcomes, prevalence of urine drug screening increased twice as much in the QI clinics (38.8% (SE 4.4) vs. 19.1% (SE 7), p = 0.020). While there were no other statistically significant differences between the QI and comparison clinics, Cohen’s d effect sizes again favored the QI clinics.
Table 4. Target Patient Population Treated with ≥ 90 mg/day of Morphine-equivalent Opioid Dose: Change in Outcomes.
|
Change in Intervention clinics
(N=271 at exit)
|
Change in Comparison clinics
(N=248 at exit)
|
Change in Intervention versus Comparison clinics
|
Characteristics
|
Change from baseline
|
p value†
|
Cohen’s d
|
Change from baseline
|
p value†
|
Cohen’s d
|
Difference
|
p value‡
|
Cohen’s d
|
Completed treatment agreement (past 12 months), % (SE),
95% CI (LB,UB)
|
41.1 (9.3)
(23.0, 59.3)
|
<0.001
|
1.9
|
40.9 (8.2)
(24.9, 56.9)
|
<0.001
|
2.0
|
0.2 (12.2)
(-23.7, 24.1)
|
0.783
|
0.0
|
Completed urine drug testing (past 12 months), % (SE),
95% CI (LB,UB)
|
38.8 (4.4)
(30.2, 47.4)
|
<0.001
|
2.3
|
19.1 (7.0)
(5.4, 32.8)
|
0.036
|
1.0
|
19.8 (10.4)
(-0.7, 40.2)
|
0.020
|
0.8
|
Completed depression screening (past 12 months), % (SE),
95% CI (LB,UB)
|
54.2 (7.6)
(39.4, 69.0)
|
<0.001
|
2.6
|
60.6 (9.7)
(41.5, 79.7)
|
<0.001
|
2.6
|
-6.3 (13.1)
(-32.1, 19.4)
|
0.528
|
0.2
|
Completed opioid misuse risk assessment, % (SE),
95% CI (LB,UB)
|
7.2 (3.8)
(-0.3, 14.7)
|
0.060
|
0.9
|
10.3 (6.6)
(-2.7, 23.3)
|
0.074
|
0.6
|
-3.1 (8.5)
(-19.7, 13.6)
|
0.582
|
0.2
|
Documented PDMP Check (past 12 months)*, % (SE),
95% CI (LB,UB)
|
56.5 (6.9)
(43.0, 70.0)
|
<0.001
|
3.8
|
45.2 (8.0)
(29.6, 60.8)
|
<0.001
|
2.3
|
11.3 (11.3)
(-10.8, 33.4)
|
0.263
|
0.4
|
Co-prescribed benzodiazepines in at least 1 out of 3 past months, % (SE),
95% CI (LB,UB)
|
-5.8 (3.5)
(-12.6, 0.9)
|
0.091
|
0.5
|
-6.0 (5.1)
(-16.0, 4.0)
|
0.039
|
0.4
|
0.1 (6.7)
(-13.0, 13.3)
|
0.486
|
0.0
|
Percentage of the adult clinic population, % (SE),
95% CI (LB,UB)
|
-0.2 (0.1)
(-0.3, 0.0)
|
0.030
|
0.4
|
-0.1 (0.0)
(-0.1, 0.0)
|
0.031
|
0.3
|
-0.1 (0.1)
(-0.2, 0.1)
|
0.224
|
0.5
|
MED (past 90 days), mg/day, mean (SE), 95% CI (LB,UB)
|
-26.9 (10.0)
(-46.5, -7.3)
|
0.007
|
0.5
|
-33.8 (16.5)
(-66.2, -1.4)
|
0.005
|
0.6
|
6.9 (21.6)
(-35.3, 49.2)
|
0.356
|
0.1
|
Legend
Population of adult patients with chronic non-cancer pain treated with opioid dose ≥ 90 morphine-equivalent mg/day in the 9 intervention and 17 comparison primary care clinics; characteristics at exit (December 2017*) and their patient-weighted change from baseline.
CI (LB,UB): Confidence Interval (Lower Bound, Upper Bound); MED: Morphine-equivalent Dose; PDMP: Prescription Drug Monitoring Program; SE: standard error
* Exit data for the PDMP outcome was collected in March 2017; starting in April 2017, the PDMP check documentation rose to approximately 100% across all clinics due to the changes in documentation requirements.
† p value was determined using the single-sample mean test comparing clinic-level changes in outcomes from baseline to exit
‡ p value was determined using the two-sample means test comparing clinic-level changes in outcomes from baseline to exit for intervention versus comparison clinic groups