Design
Consistent with the quasi-experimental two-group pre-post design, we used the difference-in-differences (DID) method to ameliorate potential confounding bias.16,17 The underlying assumption of the DID method is that the change in readmission rates from pre- to post-period in the comparison group is a good proxy of the counterfactual change in the pilot group had there been no pilot program (Figure S1 in supplemental materials). The target estimand is the average treatment effect on the treated which answers the question: for patients treated in the pilot sites, was the program a cause for the change in readmission rate? On the probability scale, a DID method estimates the difference of risk differences (DRD); and on the odds ratio (OR) scale, a DID method estimates the ratio of ORs (ROR). The pre-and post-periods for the BCN (a health maintenance organization [HMO]) patients were 7/1/2016 to 4/30/2018 and 5/1/2018 to 8/31/2019; the corresponding periods for the BCBSM (a preferred provider organization [PPO]) patients were 2/1/2017 to 11/30/2018 and 12/1/2018 to 3/30/2020. In the post periods, another site implemented a program like C.L.I.M.B.; thus, to avoid contamination patients whose initial detox occurred at that site were excluded. The pilot program was approved by the BCN and BCSM medical directors, and the current evaluation was approved by the Institutional Review Board of Michigan State University as non-human subject research (STUDY00000846).
Patients
Patients 18 years or older, who had detox inpatient stay for a diagnosis of OUD in any of the two pre-periods were included in the study. To ensure data completeness, a patient had to be enrolled in the health plan for 6 or more months before the initial detox in pre-period to capture baseline comorbidity; and the initial detox did not occur within 90-days of each period’s end date.
Interventions
Following guidelines set forth by the ASAM,13,14 the C.L.I.M.B. program included services for the continuum of OUD cycle, including detoxification, residential care, intensive outpatient program, outpatient with medically assisted treatment (MAT), and smart phone support application (app) originated in 2011 at University of Wisconsin Center for Health Enablement Support System (CHESS).called Addiction-CHESS (A-CHESS), which is a comprehensive relapse prevention tool for patients with substance use disorders (SUDs). The critical tasks in each level of care (LOC) of the pilot program are described below.
a) Detoxification: A patient will stay 1-3 days in LOC (ASAM LOC 3.3 or higher) appropriate for acute withdrawal management to stabilize the patient’s physical symptoms of withdrawal and “jump start” the MAT process. The sub-acute detoxification phase can be shorter when the continuing withdrawal can be addressed in the residential LOC.
b) Residential services: A patient can stay 7-14 days in supervised living (ASAM LOC 3.0-3.1) where there is a safe, structured recovery environment with lesser temptation to relapse and have access to medical attention if required to address stabilizing withdrawal symptoms. Critical items that will be addressed beyond biomedical conditions are education about the disease, motivation to change, tools to reduce relapse potential, e.g., the A-CHESS app, comprehensive treatment plan, family/systems assessment to identify additional members who may have SUDs, and continuation of MAT where appropriate.
c) Partial hospitalization/intensive outpatient services: This LOC (ASAM LOC 2.0-2.5) would continue the work started in the residential LOC. The patient would live at home or in supportive recovery housing, ideally be back to work or school and simultaneously attending 12 step programming. They would use the A-CHESS app to identify sober supports (sponsor, sober coach) and have routine drug testing with weekly reporting to the therapist. This LOC would continue for 12-24 sessions in 2 months on the average.
d) Outpatient services: In this LOC (ASAM LOC 1.0), which could last 12 months or more, therapists and treatment team will need to monitor the patient through A-CHESS, have routine contact with the peer support/coach, use motivational interviewing and behavioral activation skills to support continued therapy, develop benchmarks and celebrate achievement.
e) A-CHESS: The smartphone app applies the science of self-determination theory18 to help patients succeed in addiction recovery. Key A-CHESS services19 include 1) a “Help” button linked to the patient’s preapproved supporters, positive and potentially distracting games, and audio-video relaxation recording; 2) cognitive behavioral therapy boosters; 3) functionality monitoring with self-assessment tools; 4) a global positioning system location tracker that will initiate a patient-defined action (e.g., contacting sober coach) when s/he approaches a high-risk location, and 5) just-in-time feedback via a counselor dashboard.
Main Measures
Primary outcome: 90-day detox readmission after an initial detox inpatient stay at any facility. Readmission was identified by the same method as the initial detox: any inpatient stay with a diagnosis of F11.x or F11.xx using the International Classification of Diseases, 10th version, Clinical Modification (ICD-10-CM) codes, and revenue codes 01x6 (x=1, 2,3,4, or 5).
Treatment groups: the National Provider Identifier codes for the two pilot facilities were used to identify patients in the pilot group (two sites).
Level of care: MAT was identified using National Drug Codes in pharmacy claims and the Current Procedural Terminology codes; and revenue codes and/or procedure codes were used to find LOC 1.0-2.5 (partial hospitalization/intensive outpatient services/outpatient) services (list of these codes available upon request).
Comorbidity: the Agency for Healthcare Research and Quality Clinical Classification Software Refined version (v2021.2)20 based on the ICD-10-CM codes was used to find in medical claims the comorbid conditions 6 months prior to the initial detox in each period, including mood disorders; anxiety-, fear-, trauma- or stressor-related disorders; alcohol-, cannabis-, sedative-, stimulant- hallucinogen- or inhalant- related disorders, neoplasms; suicidal ideation/attempt or intentional self-harm; endocrine, nutritional and metabolic diseases; diseases of the nervous system, circulatory system, respiratory system, digestive system, musculoskeletal system, or genitourinary system. Emergency room visits in past 6 months were identified using revenue codes 045x (x=0-9).
Covariates: Patient’s age, sex, HMO or PPO plan types, and residential zip codes were extracted from health plan enrollment files. The 5-digit zip codes were linked to the census tracts using the U.S. Department of Housing and Urban Development zip code crosswalk files21 where a census tract with the highest residential ratio was chosen when multiple tracts were within the same zip code.22 Past research linked living in a disadvantaged neighborhood to worse health conditions and increased utilizations. We used the 2018 Area Deprivation Index (ADI),23,24 2015 Childhood Opportunity Index (COI),25 and 2018 Social Vulnerability Index (SVI)26 to approximate the neighborhood characteristics and as proxies to patient socioeconomic status. Higher ADI rankings represent more disadvantaged neighborhoods; higher COI scores measure more opportunities; and higher SVI scores indicate more vulnerable neighborhoods. All indices were transformed to range from 0 to 100.
Analytic Approach
We compared the differences in covariates and comorbidities between the C.L.I.M.B. and control group in the pre- and post-periods using chi-square tests for categorical variables and t-tests for continuous variables. As in the tradition for propensity score analysis, we also presented the standardized differences (difference divided by the pooled standard deviation) between the two group where absolute value greater than 0.1 were indicative of clinically or practically important differences,27 i.e., they are 0.1 standard deviations away from each other. We estimated the DID effects using six methods to triangulate evidence: 1) multivariable logistic regression adjustment (RA) for comorbidities and covariates; 2) augmented inverse probability weighted (IPW) marginal effects28 where covariates for the outcome model and the propensity scores (PS) model were selected using logistic lasso;29 3) IPW estimation where the PS were estimated using logistic regressions; 4) IPW-RA double robust method;30 5) bias-corrected single nearest neighbor matching method31 based on Mahalanobis distance matrix of the covariates; and 6) PS matching with caliper 0.2. The 95% confidence intervals (CI) were estimated using the percentile-based bootstrap CI using 1,000 bootstrapped samples. All analyses were performed in Stata version 17.32
Sensitivity Analyses
We performed two sets of sensitivity analysis. First, we excluded 123 patients (236 events) who were in both the pre- and post-periods, because the analyses may be contaminated by the correlations between observations for the same patients, especially when the patient was in different treatment groups across periods. Secondly, many randomized controlled trials (RCTs) include stringent inclusion/exclusion criteria. We applied the criteria of the MAT + A-CHESS trial19 to compare the results of C.L.I.M.B. in the community-based population versus in a trial-selected population. The trial required the patients to have no acute medical problem with immediate inpatient treatment needs, no history of psychotic disorders, willing to participate, able to read and write in English, not pregnant, willing to share health-related data with primary care physicians, and at the trial intake, abstinent from opioids for at least one week and no longer than two months, except for medications used to treat OUD. We applied as many of these criteria as possible using claims data (underlined above).