Study design and data source
We conducted a retrospective comparative new-user cohort study in the IBM® MarketScan® Commercial Claims and Encounters Database (CCAE), which primarily consists of de-identified, patient-level health data from over 142 million individuals enrolled in employer-sponsored health insurance plans in the United States. The CCAE database includes adjudicated health insurance claims (inpatient, outpatient, and prescription) and enrollment data from large employers and health plans who provide private insurance coverage. Data were standardized to the Observational Health and Data Sciences and Informatics (OHDSI) Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) version 5.3, which maps international coding systems into standard vocabulary concepts9.
The CCAE database consists of de-identified healthcare records. In the United States, retrospective analyses of the CCAE data are considered exempt from informed consent and institutional review board (IRB) approval as dictated by Title 45 Code of Federal Regulations, Part 46 of the United States, specifically 45 CFR 46.104 (d)(4).
Study population
We identified new users of ACEI and thiazide or thiazide-like diuretic monotherapy between October 1, 2015 and January 1, 2017. For each patient, we defined the index as the date of first drug exposure.
The study was limited to patients with a minimum of 365 days of continuous enrollment in the database prior to index. We required patients to have a recorded diagnosis for hypertension at or within 365 days prior to index (see Supplemental Appendix A for a list of codes used to query the database). As described in Suchard et al., new users were defined as patients whose first observed treatment for hypertension was ACEI or thiazide or thiazide-like diuretic monotherapy10. Patients with exposure to any other active ingredient listed within the five primary drug classes for the treatment of hypertension in the 2017 American College of Cardiology/American Heart Association (ACC/AHA) guidelines (i.e., ACEI, thiazide or thiazide-like diuretics, angiotensin receptor blockers, dihydropyridine calcium channel blockers, non- dihydropyridine calcium channel blockers) within 365 days prior or 7 days post-index were excluded10,11.
Example outcome of interest
We examined the safety outcome of angioedema, which was identified from diagnoses recorded on inpatient and emergency room healthcare claim records. Patients with a recorded diagnosis for angioedema at or within any time prior to index were excluded from the study.
Time-at-risk
The time-at-risk window was defined based on the intention-to-treat principle, and patients were followed from day 1 post-index to the earliest of July 31, 2019 or end of continuous observation in the database12. Analyses were limited to patients with a minimum time-at-risk of 1 day.
Patient demographic and clinical characteristics
We measured patient demographics at index including age, grouped into categories in 5-year increments; sex; and index year and month. Patient clinical characteristics included all observed condition, drug exposure, measurement and observation codes occurring within a long-term or short-term window (i.e., at or within 365 or 30 days prior to index, respectively). Furthermore, we measured all observed drug exposures occurring within the time-at-risk window. All drug exposures were grouped at both the ingredient-level and according to the Anatomical Therapeutic Chemical (ATC) classification system. Patient comorbidities were measured using the Charlson Comorbidity Index (CCI)13. Finally, we measured the following disease severity and risk scores: Diabetes Complications Severity Index (DCSI), CHADS2 score, and CHA2DS2-VASc score14,15,16. The CCI, DCSI, CHADS2 score and CHA2DS2-VASc score were measured based on all observed conditions occurring prior to the end of the time-at-risk window.
Large-scale propensity score matching
Candidate covariates were defined as all aforementioned patient demographic and clinical characteristics, and heuristic feature selection was used to identify candidate covariates with a frequency greater than 0.1%. We developed propensity models using LASSO regression with 10-fold cross-validation for hyperparameter tuning including all candidate covariates identified through heuristic feature selection, and propensity scores were calculated using the propensity model17. New users of ACEI and thiazide or thiazide-like diuretic monotherapy were matched at a 1:1 ratio using greedy matching enforcing a caliper of 0.10 and 0.20 of the pooled standard deviation of the logit of propensity scores in two separate analyses. To facilitate comparisons between CM and PSM, we defined matching covariates as candidate covariates with non-zero beta coefficients in the propensity model.
Cardinality matching
Heuristic feature selection of candidate covariates was performed as previously described with one notable exception: due to memory constraints associated with CM, in analyses using the full study population, the heuristic feature selection used a frequency threshold of 2% instead of 0.1%. Specifically, CM failed to converge to a matched sample due to insufficient memory while attempting to match on approximately 220 million data points (172,117 patients and 1,237 matching covariates). The frequency threshold used within all subsample group analyses was consistent between CM and PSM.
Matching covariates – covariates used in the CM - were empirically selected; propensity scores were estimated as previously described and matching covariates were defined as candidate covariates with non-zero beta coefficients in the LASSO propensity model. CM utilizes advancements in optimization algorithms to solve for the largest sample size meeting prespecified balance criteria (e.g., maximum standardized mean difference [SMD] of matching covariates)4. We performed CM using the following prespecified balance criteria in four separate analyses: exact marginal distributional balance (i.e., fine balance; SMD = 0) and maximum SMD of 0.01, 0.05 and 0.10 of matching covariates between study groups.
All analyses were performed using an Amazon Web Services (AWS) Virtual Private Cloud (VPCx) m4.4xlarge Elastic Compute Cloud (EC2) instance. This instance included 16 2.3 GHz Intel® Xeon® vCPUs, 64 GiB of memory and a dedicated Elastic Block Storage (EBS) bandwidth of 2000 Mbps. Furthermore, all analyses were performed in R version 3.6.3 using the Health Analytics Data-to-Evidence Suite (HADES), Gurobi™ solver and cardmatch library18.
Evaluation of post-match sample size
We evaluated patient retention in the matched samples after CM and PSM based on post-match sample size.
Evaluation of post-match covariate balance
The performance of CM and PSM were compared in terms of post-match covariate balance. In order to determine the level of balance achieved within covariates indirectly and directly adjusted during matching, candidate and matching covariate balance, respectively, were assessed separately. SMDs, as defined by Rosenbaum et. al (see Eq. 1), were used to assess the post-match balance of candidate and matching covariates; specifically,
SMD = (x̄treatment - x̄comparator) / sp
where x̄treatment and x̄comparator represent the post-match covariate mean of treatment and comparator group, respectively, and sp represents the pre-match covariate pooled standard deviation19. An absolute SMD less than 0.10 was considered balanced.
Evaluation of post-match residual confounding
Residual study bias due to unmeasured potential confounders and systematic error may still exist subsequent to CM or PSM20,21. To quantify the magnitude of residual study bias, we included a total of 105 negative control outcomes in our experiment believed to be caused by neither ACEIs nor thiazide or thiazide-like diuretics, which, therefore, have a true hazard ratio equal to 120. These negative control outcomes were identified through a data-rich algorithm and manual clinical review (see Supplemental Appendix B for a list of negative control outcomes used in the current study)22. Hazard ratios were estimated for negative control outcomes as well as the example outcome of interest (angioedema) using unconditional Cox proportional hazards models in the matched samples
Comparing the estimated hazard ratios of the negative control outcomes to the ground truth (of no effect) provides insight into residual study bias. We assume the observed log hazard ratio (^θi) depends on the log of the true effect size (θi), which is assumed to be 0, plus a systematic error component (βi), and let τi denote the standard error corresponding to θi. Furthermore, we assume βi to be distributed following a normal distribution with parameters µ and σ2, which we estimate using the observed estimates (i.e., ^θi) of negative control outcomes20. In summary, we assume:
^θi ∼ N (θi + βi, τ2i ), and
βi ∼ N (µ, σ2)
To summarize the empirical null distribution into a single measure we computed the expected systematic error (ESE), defined as the expected absolute systematic error based on the estimated null distribution parameters:
ESE = E(|βi|)
Given a finite number of negative control outcomes and uncertainty in estimated hazard ratios due to limited sample size, the distribution parameters and, therefore, the ESE come with uncertainty, which we quantified using Markov-Chain Monte Carlo and expressed as 95% credible intervals.
Analyses of angioedema outcome
Unconditional Cox proportional hazards models were used to compare the safety outcome of angioedema between study groups in the matched samples. All hazard ratio (HR) estimates, 95% confidence intervals (CI) and p-values were calibrated to incorporate the uncertainty expressed in the empirical null distribution of negative control outcomes20,23. We considered a two-sided calibrated p-value < 0.05 to be statistically significant. For reference, we further examined uncalibrated effect estimates.
Analyses of subsample groups
All aforementioned analyses, with the exception of analyses of the angioedema outcome, were repeated in a series of progressively smaller subsample groups, including a 10%, 1% and 0.5% subsample group. The 10%, 1% and 0.5% subsample groups included 5, 50 and 100 subsample draws, respectively. Each subsample draw was performed by random sampling without replacement from the study population stratified by study comparison group.
Within each subsample draw, candidate covariates were defined as all aforementioned patient demographics and clinical characteristics observed within the respective subsample draw, and filtered using the aforementioned frequency thresholds. Propensity scores were estimated within each draw as previously described, and matching covariates were defined as those candidate covariates with non-zero beta coefficients in the propensity model for that draw. As such, a distinct set of candidate covariates and matching covariates were identified for each subsample draw; however, within individual subsample draws, candidate and matching covariates were consistent between CM and PSM.
For each subsample group, we assessed the average post-match sample size of their respective subsample draws. Meanwhile, candidate and matching covariate balance were assessed based on the SMD of covariates across all subsample draws within each subsample group considered jointly. Hazard ratios for negative control outcomes were estimated independently within each subsample draw using unconditional Cox proportional hazards models. Residual confounding was assessed based on the ESE of the empirical null distribution of negative control outcomes, which was derived from hazard ratio estimates considered jointly across all subsample draws within each subsample group. Analyses of the angioedema outcome were not performed across subsample groups due to insufficient occurrence of the outcome.