Assessing the external validity of the VALIDATE-SWEDEHEART trial

doi:10.21203/rs.3.rs-25041/v1

Download PDF

Research article

Assessing the external validity of the VALIDATE-SWEDEHEART trial

https://doi.org/10.21203/rs.3.rs-25041/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 20 May, 2021

Read the published version in Clinical Trials →

Version 1

posted

You are reading this latest preprint version

Background The VALIDATE-SWEDEHEART trial was a registry-based randomized trial comparing bivalirudin and heparin in patients with acute myocardial infarction undergoing percutaneous coronary intervention. It showed no differences in mortality at 30 or 180 days. This study examines how well the trial population results may generalize to all of the screened patients with fulfilled inclusion criteria for mortality at 30 and 180 days.

Methods The standardized difference in the mean propensity score for trial inclusion between the trial population and the screened not-enrolled with fulfilled inclusion criteria population was calculated as a metric of similarity. A predefined criterion of ≤ 0.25 was used. Predictions were futher used to standardize the observed trial results to all the screened patients with fulfilled inclusion criteria through an inverse-probability weighted Cox regression analysis, thereby evaluating the generalizability of the trial results to this population. An adjusted Cox regression analysis was applied to analyze differences in mortality with respect to anticoagulant treatment between the trial population and the screened not enrolled with fulfilled inclusion criteria.

Results The propensity-score difference was 0.61, above the limit defining similar populations. There were no significant differences in mortality between bivalirudin and heparin in the inverse-probability weighted analysis [HR 1.11, 95% CI (0.73 1.68)]. In the interaction analysis, the neutral effect of bivalirudin vs. heparin remained in the screened not-enrolled with fulfilled inclusion criteria population [HR 1.01, 95% CI (0.63 1.61)].

Conclusion The propensity-score difference demonstrated that the screened not enrolled with fulfilled inclusion criteria and the trial population were not similar. The inverse-probability weighted analysis showed no significant differences in mortality associated with treatment in either of the populations as was reiterated by the Cox analysis. From these analyses, we conclude that the VALIDATE trial results may be generalized.

Health Economics & Outcomes Research

VALIDATE-SWEDEHEART

bivalirudin

heparin

anticoagulants

antithrombotics

percutaneous coronary intervention

The results of randomized controlled trials (RCT) are usually the basis on which medications are approved for public use. Efficacy trials primarily use a homogeneous set of participants to ascertain whether an intervention is successful under optimal conditions.¹ Primary focus is on internal validity, i.e. the legitimacy of the study in terms of design and limiting bias.² Less attention has been given to external validity³, which refers to the extent to which results of a randomized trial can be generalized to a real-world population.

The VALIDATE-SWEDEHEART trial was a registry-based randomized trial comparing bivalirudin and heparin for patients who underwent PCI without planned GPIIb/IIIA after acute myocardial infarction (AMI). It showed no differences in the composite endpoint at 180 days, which included death from any cause, acute myocardial infarction or bleeding. It showed no differences in the secondary endpoint of death either, which is the endpoint of focus for this study. The external validity of VALIDATE should be further examined as VALIDATE had an open-label design in which participating physicians may have been biased, and the trial population may not have been representative of all patients undergoing percutaneous coronary intervention (PCI). There are no accepted guidelines for evaluating external validity in randomized controlled trials⁴, and many conditions may affect external validity, including the trial setting as compared to routine practice, patient selection, the characteristics of the trial participants, differences between the trial protocol and clinical practice, and the inclusion and exclusion criteria.⁵

The external validity of trials related to AMI, PCI, and antithrombotic medications have not been studied in patients who fulfilled inclusion criteria upon screening, but were not enrolled in the trial. The SWEDEHEART registry provides a unique opportunity to conduct such research. We investigated the difference between the screened not enrolled with fulfilled inclusion criteria (SNEF) and the trial population (TP) in terms of 30 and 180 days mortality to assess the correspondence of results to those of the VALIDATE-SWEDEHEART trial. Additionally, we investigated what the trial results would have been, had it included all screened patients with fulfilled inclusion criteria (SFC).

Trial population

The VALIDATE-SWEDEHEART (bivalirudin versus heparin in ST-Segment and Non-ST-Segment Elevation Myocardial Infarction Registry Trial) was a multicentre, controlled, registry-based randomized clinical trial comparing treatment effects between bivalirudin and heparin in conjunction with the PCI for AMI.^{6, 7} A composite of death from any-cause, AMI and bleeding at 180 days was the primary endpoint. Twenty-five of Sweden´s 29 PCI centers participated in the trial. Of the 12561 patients screened for participation, 6555 were not randomized and 6006 (48%) were randomized and referred to in this paper as the trial population (TP).

Screened not enrolled

Of the 6555 non-randomized patients, 150 had a missing treatment assignment in the registry and were excluded, leaving 6405 in the screened not-enrolled (SNE) population, a secondary population of interest in this study. Of those 6405, 3422 did not meet the inclusion criteria for various reasons: not able to give informed consent (22%), not treated with ticagrelor or prasugrel (21%), indicated as other reason for non-inclusion in the registry (19%), 5000 U of heparin before arrival or 3000 U in the lab before angiography (13%), ST-segment elevation myocardial infarction/non-ST-segment elevation myocardial infarction (STEMI/NSTEMI) (9%), reduced kidney function (5%), planned glycoprotein IIb/IIIa inhibitors (GP) (3%), life expectancy of one year or less (3%), contraindication for either heparin or bivalirudin in the trial (2%), continuous bleeding (2%), uncontrolled hypertension, thrombocytopenia, and endocarditis (0.9%), not being 18 years old (0.1%). Thus, 2983 remained as the screened not enrolled population with fulfilled inclusion criteria (SNEF). The reason for the SNEF not being randomized was not documented. The VALIDATE-SWEDEHEART trial was powered for evaluation of STEMI and NSTEMI separately with enrolment of an equal number of each type of ACS. Thus 3001 patients with NSTEMI and 3006 STEMI were enrolled. In the general SWEDEHEART population however, there were more NSTEMI then STEMI, which made the SNEF even more unbalanced when it came to the number of NSTEMI compared to STEMI. Additionally, some of the hospitals did not include patients during off-hours, i.e. night shifts or the weekends and some physicians did not participate in the study.

The study populations

The two patient populations were combined into a single dataset for further analysis, 6006 randomized in the VALIDATE trial and 2983 with known treatment assignment and satisfying inclusion criteria. One assumption, in order to assess the generalizability of trial results, was to remove those who had zero chance of being selected to participate in the trial.⁸ Therefore, those patients who did not meet the inclusion criteria were removed, leaving the (SNEF) (Figure 1).

In total, 8989 patients were included in this study. Due to the randomization, the TP was balanced on the observed covariates; the SNEF patients were not randomized to bivalirudin or heparin, thus treatment groups were not balanced with respect to the observed covariates (Table 1).

An additional analysis of all 12411 screened patients with known treatment assignments was also performed (Appendix).

Endpoints

In the VALIDATE study (TP), bleeding events were adjudicated and classified by an endpoint committee, but bleeding events were not adequately reported in the SWEDEHEART registry for the SNEF in the present study. Thus, an analysis with the composite endpoint of death, AMI or bleeding was not possible. We therefore used death at 30 and 180 days as the primary endpoint.

Methods

A baseline table, Kaplan-Meier curves, and log-rank test were produced to describe the study data (Table 1, Figures 2-4).

A propensity score method, using logistic regression with inclusion to VALIDATE as the outcome to generate scores, was used to evaluate population similarity with respect to prognostic factors between TP and the SNEF.⁸ This was done in a stratified analysis of STEMI and NSTEMI as well. Traditionally, propensity scores are used to mimic randomized trials with observational data through balancing observed covariates between treatment groups in observational studies.⁹ The lower the propensity-score difference, the more similar the populations. Standardized mean differences exceeding 0.25 were considered large and indicated dissimilarity between populations as defined by Stuart et al.⁸ Logistic regression modelled the probability of being selected in the VALIDATE trial with variables of interest. These covariates were potentially associated with differences in VALIDATE trial selection. Additional variables were included to ensure balance.¹⁰ The model included the following variables: STEMI/NSTEMI, age, sex, low weight, smoking, diabetes, hypertension, hyperlipidaemia, previous AMI, previous PCI, previous coronary artery bypass graft (CABG), previous stroke, renal failure, thrombectomy, ticagrelor before inclusion, clopidogrel before inclusion, cardiopulmonary resuscitation (CPR), puncture location, creatinine (umol/L), ACE inhibitors, Killip class and off-hours vs. regular hours (Appendix). Off-hours, signified the night or weekend shifts. Incidence proportions for off-hours and regular hours were calculated for the different groups and endpoints. Variables that had 3% or more missing were altered to include an extra category for missing; these variables were smoking, CPR, ACE inhibitors, Killip class and off-hours vs. regular hours.

Propensity scores were then used in an inverse-probability weighted (IPW) Cox regression analysis using the TP only to estimate the difference in mortality (DIM) as it would have been had the trial included the SFC.⁷ The point of the weights is to weight down patients who were very likely to be included as well as weight up those who had a very low probability of being in the trial. The patients with a very low probability of being in the trial were rare and most comparable to the non-randomized population¹¹ in terms of the covariates included in the propensity score. A person with a high probability to be included in the trial who had a large propensity score received a weight equal to the inverse of their propensity score, a low weight. If the patient’s propensity score were 0.8, then their weight would be 1/0.8=1.25. A person who was very unlikely to be included in the trial, but was included anyway would receive a larger weight; a person with a propensity score of 0.2 would, therefore, have a weight of five. These weights were checked via trimmed and stabilized weighting.¹²

All the variables that were included in the IPW analysis were checked in separate Cox models as part of an interaction term with the treatment variable. A sensitivity analysis solely including the significant variables from this subgroup analysis were produced to see if the mean propensity score difference changed.

To investigate potential differences between the TP and the SNEF in terms of mortality, a Cox regression with a treatment x trial selection interaction term was applied. The inclusion of the interaction term helped produce hazard ratios (HR) between treatments for the SNEF and TP separately as well as the HR for the treatment effect in the SNEF divided by the corresponding HR in the TP. The interaction term also produced HR’s comparing mortality across treatments, i.e. heparin treatment in the SNEF compared to heparin treatment in the TP. The same was produced for bivalirudin vs. bivalirudin.

The analysis was adjusted for: STEMI/NSTEMI, age, gender, low weight, smoking, diabetes, hypertension, hyperlipidaemia, previous AMI, previous PCI, CABG, previous stroke, renal failure, thrombectomy, ticagrelor before trial, clopidogrel before trial, CPR, puncture location, creatinine (umo/L), ACE inhibitors, Killip class and regular vs. off-hours. We refer to this analysis as the adjusted Cox model (Tables 2-3).

These analyses were repeated as a sensitivity analysis with separate models for STEMI and NSTEMI populations to see if the treatment effects changed by type of myocardial infarction.

All analyses were performed in STATA/MP version 15. All tests were two-sided and the alpha-level was set to 0.05.

Inclusion in VALIDATE analysis

The standardized propensity-score difference of being included to VALIDATE was 0.61 and therefore did not meet the criterion for similarity of ≤ 0.25. When including only significant variables from the sensitivity subgroup analysis of DIM, the propensity-score difference was reduced to 0.42. The significant variables were STEMI/NSTEMI, age, CABG, previous stroke, renal failure, puncture site, and creatinine level. When the differences were stratified by type of AMI, the results were 0.51 for STEMI and 0.58 for NSTEMI.

Mortality at 30 days

The incidence of all-cause mortality was 1.8% in the TP, 3.8% in the SNEF, and 5.3% in the SNE.

The incidence of death for off-hours and regular hours was, respectively, 2.4% and 1.4% in the TP; 5.1% and 3.0% in the SNEF, and 7.9% and 3.8% in the SNE.

Mortality with bivalirudin and heparin treatment was, respectively, 1.9% and 1.7% in the TP, 5.5% and 3.1% in the SNEF, and 7.2% and 4.6% in the SNE.

The log-rank test showed significant differences in the survival curves for the SNEF in favour of heparin, (P = .01), but not in the trial population, (P = 0.65). For the TP there were no significant DIM for STEMI (P = 0.69) or NSTEMI (P = 0.81). The results in the SNEF were similar for STEMI (P = 0.60) and NSTEMI (P = 0.84). (Figs. 2–4).

There were no significant differences in mortality between treatment groups in the IPW analysis of the TP, which were re-weighted to be closely aligned to the pooled TP and SNEF [HR 1.11, 95% CI (0.73 1.68)].

The adjusted Cox model revealed no significant DIM between bivalirudin and heparin in the SNEF [HR 1.01, 95% CI (0.63, 1.61)]. There were no significant differences in the HR for bivalirudin vs heparin between the two populations [HR 0.90, 95% CI (0.48, 1.66)], i.e. the hazard ratio between bivalirudin and heparin in the SNEF divided by the hazard ratio between bivalirudin and heparin of the TP (Table 2).

The Cox model showed significantly higher mortality for SNEF compared to the TP for heparin vs. heparin [HR 1.55, 95% CI (1.01 2.37)], but not for bivalirudin vs. bivalirudin [HR 1.39, 95% CI (0.88 2.20)] The differences with only STEMI or only NSTEMI were all non-significant (Table 2).

Mortality at 180 Days

Mortality was 2.9% in TP, 5.4% in SNEF and 8.1% in SNE.

Mortality for off-hours and regular hours was 3.6% and 2.4% in TP; 6.7% and 4.6% in SNEF, and 10.7% and 6.5% in SNE, respectively.

Mortality in in bivalirudin and heparin treatments was 3.0% and 2.8% in TP, 5.5% and 3.1% in the SNEF, and 10.0% and 7.4% in the SNE, respectively.

There were no significant differences in treatment in the IPW analysis of TP [HR 0.97, 95% CI (0.70 1.35)].

There were no significant DIM between bivalirudin and heparin in the SNEF at 180 days [HR 1.03, 95% CI (0.70, 1.51)] in the adjusted Cox model. There were no significant differences between the hazard ratios SNEF vs. TP, [HR 1.02, 95% CI (0.62, 1.68)]. The adjusted model did not show significantly higher mortality for SNEF compared to TP for heparin vs. heparin [HR 1.31, 95% CI (0.94 1.82)] or bivalirudin versus bivalirudin [HR 1.33, 95% CI (0.91 1.95)]. We found no differences with only STEMI and only NSTEMI (Table 3).

The entire screened population, including those not fulfilling the inclusion criteria, was not the objective of this study, but in general the results are in agreement. See Appendix.

Table 1

Baseline table of TP and SNEF
Characteristics	TP (n = 6006)	SNEF (n = 2983)	TP Bivalirudin (n = 3004)	TP Heparin (n = 3002)	SNEF Bivalirudin (n = 853)	SNEF Heparin (n = 2130)
STEMI - no. (%)	3005 (50.0)	1049 (35.2)*	1501 (50.0)	1504 (50.1)	567 (66.5)	482 (22.6)**
Male sex - no. (%)	4406 (73.4)	2092 (70.1)*	2229 (74.2)	2177 (72.5)	601 (70.5)	1491 (70.0)
Age Median - yr IQR - yr ≥ 65 no. (%)	68.0 60.0–75.0 3671 (61.1)	70.0* 60.0–78.0 1963(65.8)	68.0 59.0–75.0 1819 (60.6)	68.0 60.0–75.0 1852 (61.7)	67.0 57.0–76.0 494 (57.9)	70.0* 62.0–78.0 1469 (69.0)
BMI Median IQR	26.9 24.5–29.7	26.9 24.3–30.1	26.8 24.5–29.7	26.9 24.5–29.7	27.0 24.2–30.2	26.9 24.3–30.1
Weight < 60 kg - no. (%)	295 (4.9)	209 (7.0)*	139 (4.6)	156 (5.2)	68 (8.0)	141 (6.6)
Previous Smoker - no. (%)	2047 (34.1)	1032 (34.6)*	1027 (34.2)	1020 (34.0)	270 (31.7)	762 (35.8) **
Current Smoker - no. (%)	1426 (23.7)	650 (21.8)*	716 (23.8)	710 (23.7)	218 (25.6)	432 (20.3)**
Diabetes - no. (%)	999 (16.6)	672 (22.5)*	491 (16.3)	508 (16.9)	154 (18.1)	518 (24.3)**
Hypertension - no. (%)	3105 (51.7)	1728 (57.9)*	1557 (51.8)	1548 (51.6)	426 (49.9)	1302 (61.3)**
Hyperlipidaemia - no. (%)	1889 (31.5)	1195 (40.1)*	953 –(31.7)	936 (31.2)	271 (31.8)	924 (43.4)**
Previous AMI - no. (%)	974 (16.2)	739 (24.8)*	490 (16.3)	484 (16.1)	168 (19.7)	571 (26.8)**
Previous PCI - no. (%)	882 (14.7)	593 (19.9)*	456 (15.2)	426 (14.2)	123 (14.4)	470 (22.1)**
Previous CABG - no. (%)	293 (4.9)	268 (9.0)*	152 (5.1)	141 (4.7)	52 (6.1)	216 (10.1)**
Previous stroke - no. (%)	240 (4.0)	145 (4.9)*	115 (3.8)	125 (4.2)	38 (4.5)	107 (5.0)**
CPR - no. (%)	46 (0.8)	33 (1.1)*	26 (0.9)	20 (0.7)	14 (1.6)	19 (0.9)**
Killip class II, III and IV - no. (%)	194 (3.2)	169 (5.7)*	108 (3.6)	86 (2.9)	84 (9.9)	85 (4.0)**
Puncture site femoral vein - no. (%)	570 (9.5)	565 (18.9)*	290 (9.7)	280 (9.3)	204 (23.9)	361 (17.0)**
Ace inhibitor on admission - no. (%)	990 (16.5)	586 (19.6)*	501 (16.7)	489 (16.3)	139 (16.3)	447 (21.0)**
Off hours - no. (%)	576 (9.6)	226 (7.6)*	288 (9.6)	288 (9.6)	23 (2.7)	203 (9.5)**
Creatinine-median (micromole/L) IQR	80 (68–93)	80 (68–95)*	80 (68–93)	79 (68–92)	79 (66–94)	80 (68–95)**

Table 2

Adjusted model with interaction (Treatment and Inclusion to Validate) mortality at 30 days
AMI - acute myocardial infarction PCI - percutaneous coronary intervention CABG - previous coronary artery bypass graft CPR - cardiopulmonary resuscitation * significant differences between TP and SNEF ** significant differences between bivalirudin and heparin in SNEF

TP and SNEF

Interaction model

TP event %

Bivalirudin Heparin, HR Bivalirudin vs. Heparin

SNEF event %

Bivalirudin Heparin, HR

Bivalirudin vs. Heparin *

SNEF HR Bivalirudin vs. Heparin

SNEF vs. TP

Ratio of HR

Bivalirudin vs. Heparin

SNEF vs. TP

HR Heparin vs. Heparin and HR Bivalirudin vs. Bivalirudin

All patients

n = 8989

1.9% vs. 1.7%

HR 1.13

CI (0.75 1.68)

P = 0.562

5.5% vs. 3.1%

HR 1.80

CI (1.24 2.62)

P = 0.002

HR 1.01

CI (0.63 1.61)

P = 0.971

HR 0.90

CI (0.48 1.66)

P = 0.728

HR 1.55

CI (1.01 2.37)

P = 0.043

HR 1.39

CI (0.88 2.20)

P = 0.157

STEMI

n = 4054

2.9% vs. 2.6%

HR 1.03

CI (0.64 1.66)

P = 0.906

6.9% vs. 7.9%

HR 0.87

CI (0.56 1.36)

P = 0.534

HR 0.84

CI (0.48 1.47)

P = 0.538

HR 0.82

CI (0.39 1.70)

P = 0.586

HR 1.56

CI (0.92 2.66)

P = 0.102

HR 1.27

CI (0.75 2.16)

P = 0.366

NSTEMI

N = 4935

0.9% vs. 0.9%

HR 1.27

CI (0.58 2.76)

P = 0.545

2.8% vs. 1.7%

HR 1.65

CI (0.75 3.62)

P = 0.210

HR 1.70

CI (0.72 4.03)

P = 0.226

HR 1.34

CI (0.42 4.27)

P = 0.620

HR 1.60

CI (0.78 3.27)

P = 0.198

HR 2.14

CI (.84 5.47)

P = 0.111

Table 3

Adjusted model with interaction (Treatment and Inclusion to Validate) mortality at 180 days
* Interaction term only (Treatment and Inclusion to Validate) HR - Hazard ratio CI − 95% Confidence interval P - p-value

TP and SNEF

Interaction model

TP event %

Bivalirudin Heparin, HR Bivalirudin vs. Heparin

SNEF event %

Bivalirudin Heparin, HR

Bivalirudin vs. Heparin *

SNEF HR Bivalirudin vs. Heparin

SNEF vs. TP

Ratio of HR

Bivalirudin vs. Heparin

SNEF vs. TP

HR Heparin vs. Heparin and HR Bivalirudin vs. Bivalirudin

All patients

n = 8989

3.0% vs. 2.8%

HR 1.01

CI (0.73 1.38)

P = 0.972

7.0% vs. 4.7%

HR 1.51

CI (1.10 2.08)

P = 0.012

HR 1.03

CI (0.70 1.51)

P = 0.898

HR 1.02

CI (0.62 1.68)

P = 0.939

HR 1.31

CI (0.94 1.82)

P = 0.113

HR 1.33

CI (0.91 1.95)

P = 0.140

STEMI

n = 4054

4.0% vs. 3.8%

HR 0.93

CI (0.63 1.38) p = 0.716

8.8% vs. 9.8%

HR 0.90

CI (0.60 1.34)

P = 0.596

HR 0.93

CI (0.57 1.50)

P = 0.756

HR 1.00

CI (0.53 1.86)

P = 0.992

HR 1.35

CI (0.85 2.12)

P = 0.200

HR 1.34

CI (0.86 2.10)

P = 0.197

NSTEMI

N = 4935

2.0% vs. Hep 1.9%

HR 1.09

CI (0.64 1.86)

P = 0.757

3.5% vs. 3.3%

HR 1.07

CI (0.55 2.10)

P = 0.841

HR 1.08

CI (0.52 2.23)

P = 0.835

HR 1.00

CI (0.40 2.45)

P = 0.987

HR 1.31

CI (0.80 2.14)

P = 0.281

HR 1.30

CI (.60 2.83)

P = 0.508

* Interaction term only (Treatment and Inclusion to Validate)

HR - Hazard ratio

CI − 95% Confidence interval

P - p-value

The current study compares the results from a registry-randomized trial to real-world data. We found significant differences in the unadjusted failure curves in SNEF at 30 days, with heparin being advantageous over bivalirudin. This could be explained by the fact that heparin was more frequently administered to less vulnerable patients.¹³ These differences were greatly attenuated, however, in the adjusted models.

The SNEF is a sicker group than the TP, which may be attributed to selection bias on the part of the investigators not choosing to randomize sicker patients. The standardized mean difference for inclusion to VALIDATE was 0.61, but was reduced somewhat to 0.51 and 0.58 when STEMI and NSTEMI were stratified. This conveyed that large differences remained between those who were randomized and those who were not even with stratification. There were however, no differences between bivalirudin and heparin in the IPW analysis, which may indicate that these differences do not affect the generalization of trial results to SFC. The adjusted Cox model did not exhibit significant differences either, potentially indicating a lack of treatment preference for sicker excluded patients given adjustment variables.

Other SWEDEHEART registry-randomized controlled trials like DETO2X¹⁴ and TASTE¹⁵ have not performed additional analysis of the non-randomized populations, so this type of study is a novel addition to SWEDEHEART in the registry-randomized controlled trial setting. These types of studies are a possibility for DETO2X¹⁴ and TASTE¹⁵ as a complimentary addition to gain a better understanding of how results may change in other populations.

An observational SWEDEHEART study¹⁶ that included patients from January 2007 to December 2014 and all PCI's conducted in Sweden found significantly higher 30-day mortality in the heparin group in the adjusted analysis [HR 1.23, 95% CI (1.05 1.44)], but not in the propensity-matched analysis [HR 1.20, 95% CI (0.97 1.49)].¹⁶ The VALIDATE-SWEDEHEART trial found no differences between bivalirudin and heparin at thirty days [HR 1.10, 95% CI (0.75 1.60)]. The source of the discrepancy observed in this previous trial compared to VALIDATE-SWEDEHEART may be the inclusion of patients from an earlier time-span and centres lacking experience with bivalirudin. The advantage of our study is that it is more recent with the same time-span as the VALIDATE-SWEDEHEART study, and staff at all included hospitals were experienced in providing both tested treatments, which may eliminate some bias.

Outside of SWEDEHEART, the MATRIX trial found significant differences in mortality between bivalirudin and heparin at 30 days in favour of bivalirudin for all-cause mortality as well as cardiac causes in STEMI and NSTEMI patients, but they allowed planned GP in the heparin group.^{17, 18} The HORIZONS-AMI trial also found bivalirudin to be superior to heparin and GP for death and cardiac specific causes of death at thirty days in stable coronary artery disease and NSTEMI patients.¹⁹ The HEAT-PPCI study included 97% of available STEMI patients thereby encompassing a broad spectrum of patients with delayed consent approval. There were no significant differences between bivalirudin and heparin with bailout GP for death at 28 days.²⁰ One could argue that the results from HEAT-PPCI are based on a representative sample of real-world data albeit only STEMI patients, which makes a direct comparison to the previous two studies difficult. It does however make it comparable to the STEMI patients in both the SNEF and SNE analysis in this study. We can conclude that the results of both these analysis for STEMI patients coincide with the results from the HEAT-PPCI trial of no differences between bivalirudin and heparin.

Studies on real-world data are paramount as an addition to randomized trials, although there is no consensus on what real-world data is exactly.²¹ A carotid artery study based on the National Cardiovascular Database Registry–Carotid Artery Revascularization and Endarterectomy (NCDR) Registry found large discrepancies between participants in the postmarketing study and those who were not included.²² Using the NCDR CathPCI Registry, a comparison of patients’ baseline demographics and outcomes who were included in the dual antiplatelet therapy (DAPT) participating hospitals and those who were not was performed. Those who were not included in DAPT had on average significantly longer hospital stays and multi-vessel disease.²³

A limitation in this study is the inconsistency in the meaning of treatment between the TP and SNEF, which adds bias to the adjusted Cox analysis. In the TP, it is random, but in the SNEF, there is a physician using their judgment to choose the best treatment for their patient. The additional variables in the adjusted analysis possibly remove some of this bias because their information aided in the physician's decision-making process. This bias is not present in the IPW-analysis.

Even though prospective studies are more tedious when it comes to design and confounders, they provide vital information.²⁴ The current study contributes to the literature on real-world data and echoes concerns that those who are included in randomized studies are not a representative sample of all patients, even in a registry setting. This study aimed to address the question of generalizability; it showed that there were no differences found between bivalirudin and heparin in the IPW analysis or adjusted Cox model for mortality at 30 or 180 days in the population of screened patients fulfilling inclusion criteria. The external validity of VALIDATE-SWEDHEART has been assessed.

Ethics approval and consent to participate

This study follows the Helinsiki declaration.

Yes, approved by the Ethics committee in Lund, Sweden. (N Engl J Med 2017; 377:1132-1142)

In the VALIDATE population informed signed consent was obtained. In the SWEDEHEART registry population patients are informed of being part of the registry and have the option to opt out.

Consent for publication

Not applicable

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

The funders gave scientific grants that were used for salaries to researchers and materials needed for the analyses in the manuscript.

The funders were the Swedish Research council, Swedish HeartLung foundation, and Knut and Alice Wallenberg Foundation

Authors' contributions

RR performed the data management, analysis and wrote the manuscript.

PW helped with the study design, statistical theory and interpretation of the results.

EO provided knowledge of the original VALIDATE study, revised the manuscript and cardiological expertise.

CH provided knowledge of the original VALIDATE study, revised the manuscript and cardiological expertise.

SJ provided knowledge of the original VALIDATE study, revised the manuscript and cardiological expertise.

SK provided knowledge of the original VALIDATE study and cardiological expertise.

DE provided knowledge of the original VALIDATE study, cardiological expertise, revised the manuscript and interpretation of the results.

All authors read and approved the final manuscript.

Acknowledgements

Not applicable

Authors' information

Rebecca Rylance, corresponding author Department of Cardiology, Clinical Sciences, Lund University and Skåne University Hospital, Lund, Sweden, [email protected], MSSC

+46137658 Fax

+46708607781 number

Philippe Wagner, Center for Clinical Research, Västerås, Sweden, [email protected], MSC PhD

Elmir Omerovic, Department of Molecular and Clinical Medicine at Institute of Medicine, Sahlgrenska University hospital, Gothenburg, Sweden, [email protected], MD, PhD, Head, Dept of Cardiology Gothenburg

Claes Held, Department of Medical Sciences and Cardiology, Uppsala Clinical Research Center, Uppsala University, Uppsala, Sweden, [email protected], MD, PhD, Professor Cardiology Uppsala

Stefan James, Department of Medical Sciences and Cardiology, Uppsala Clinical Research Center, Uppsala University, Uppsala, Sweden, [email protected]@ucr.uu.se, MD, PhD, Professor Cardiology Uppsala

Sasha Koul, Department of Cardiology, Clinical Sciences, Lund University and Skåne University Hospital, Lund, Sweden, [email protected], MD, PhD

David Erlinge, Department of Cardiology, Clinical Sciences, Lund University and Skåne University Hospital, Lund, Sweden, [email protected], MD, PhD, FESC, FACC
Head, Dept of Cardiology Lund
Past President of the Swedish Society of Cardiology

Marchand E, Stice E, Rohde P, Becker CB. Moving from efficacy to effectiveness trials in prevention research. Behav Res Ther. 2011;49(1):32–41.
Akobeng AK. Assessing the validity of clinical trials. J Pediatr Gastroenterol Nutr. 2008;47(3):277–82.
Dekkers OM, von Elm E, Algra A, Romijn JA, Vandenbroucke JP. How to assess the external validity of therapeutic trials: a conceptual approach. Int J Epidemiol. 2010;39(1):89–94.
Rothwell PM. External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet. 2005;365(9453):82–93.
Paul M, Bronstein E, Yahav D, Goldberg E, Bishara J, Leibovici L. External validity of a randomised controlled trial on the treatment of severe infections caused by MRSA. BMJ Open. 2015;5(9):e008838.
Erlinge D, Koul S, Omerovic E, Frobert O, Linder R, Danielewicz M, et al. Bivalirudin versus heparin monotherapy in non-ST-segment elevation myocardial infarction. Eur Heart J Acute Cardiovasc Care. 2018:2048872618805663.
Erlinge D, Koul S, Eriksson P, Schersten F, Omerovic E, Linder R, et al. Bivalirudin versus heparin in non-ST and ST-segment elevation myocardial infarction-a registry-based randomized clinical trial in the SWEDEHEART registry (the VALIDATE-SWEDEHEART trial). American heart journal. 2016;175:36–46.
Stuart EA, Cole SR, Bradshaw CP, Leaf PJ. The use of propensity scores to assess the generalizability of results from randomized trials. J R Stat Soc Ser A Stat Soc. 2001;174(2):369–86.
Austin PC. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behav Res. 2011;46(3):399–424.
Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34(28):3661–79.
Pezzi A, Cavo M, Biggeri A, Zamagni E, Nanni O. Inverse probability weighting to estimate causal effect of a singular phase in a multiphase randomized clinical trial for multiple myeloma. BMC Med Res Methodol. 2016;16(1):150.
Harder VS, Stuart EA, Anthony JC. Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychol Methods. 2010;15(3):234–49.
Zeymer U, Rao SV, Montalescot G. Anticoagulation in coronary intervention. European heart journal. 2016;37(45):3376–85.
Hofmann R, Svensson L, James SK. Oxygen Therapy in Suspected Acute Myocardial Infarction. The New England journal of medicine. 2018;378(2):201-2.
Frobert O, Lagerqvist B, Olivecrona GK, Omerovic E, Gudnason T, Maeng M, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. The New England journal of medicine. 2013;369(17):1587–97.
Grimfjard P, Erlinge D, Koul S, Lagerqvist B, Svennblad B, Varenhorst C, et al. Unfractionated heparin versus bivalirudin in patients undergoing primary percutaneous coronary intervention: a SWEDEHEART study. EuroIntervention. 2017;12(16):2009–17.
Valgimigli M, Frigoli E, Leonardi S, Rothenbuhler M, Gagnor A, Calabro P, et al. Bivalirudin or Unfractionated Heparin in Acute Coronary Syndromes. The New England journal of medicine. 2015;373(11):997–1009.
Leonardi S, Frigoli E, Rothenbuhler M, Navarese E, Calabro P, Bellotti P, et al. Bivalirudin or unfractionated heparin in patients with acute coronary syndromes managed invasively with and without ST elevation (MATRIX): randomised controlled trial. BMJ. 2016;354:i4935.
Stone GW, Witzenbichler B, Guagliumi G, Peruga JZ, Brodie BR, Dudek D, et al. Bivalirudin during primary PCI in acute myocardial infarction. The New England journal of medicine. 2008;358(21):2218-30.
Shahzad A, Kemp I, Mars C, Wilson K, Roome C, Cooper R, et al. Unfractionated heparin versus bivalirudin in primary percutaneous coronary intervention (HEAT-PPCI): an open-label, single centre, randomised controlled trial. Lancet. 2014;384(9957):1849–58.
Makady A, de Boer A, Hillege H, Klungel O, Goettsch W. What Is Real-World Data? A Review of Definitions Based on Literature and Stakeholder Interviews. Value Health. 2017;20(7):858–65.
Yeh RW, Kennedy K, Spertus JA, Parikh SA, Sakhuja R, Anderson HV, et al. Do postmarketing surveillance studies represent real-world populations? A comparison of patient characteristics and outcomes after carotid artery stenting. Circulation. 2011;123(13):1384–90.
Yeh RW, Czarny MJ, Normand SL, Kereiakes DJ, Holmes DR Jr, Brindis RG, et al. Evaluating the generalizability of a large streamlined cardiovascular trial: comparing hospitals and patients in the dual antiplatelet therapy study versus the National Cardiovascular Data Registry. Circ Cardiovasc Qual Outcomes. 2015;8(1):96–102.
Junghans C, St John M. Making it back to base: how recruitment to cohort studies affects outcomes. Eur Heart J Qual Care Clin Outcomes. 2018;4(1):6–7.

RCT randomized controlled trials

AMI acute myocardial infarction

PCI percutaneous coronary intervention

SNEF screened not enrolled with fulfilled inclusion criteria.

TP trial population

SNE screened not enrolled

SFC all screened patients with fulfilled inclusion criteria

STEMI ST-segment elevation myocardial infarction

NSTEMI non-ST-segment elevation myocardial infarction

GP planned glycoprotein IIb/IIIa inhibitors

IPW inverse-probability weighted

DIM difference in mortality

HR hazard ratios

CABG previous coronary artery bypass graft

CPR cardiopulmonary resuscitation

Download PDF

Journal Publication

published 20 May, 2021

Read the published version in Clinical Trials →

Version 1

posted

You are reading this latest preprint version

Assessing the external validity of the VALIDATE-SWEDEHEART trial

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Results

Inclusion in VALIDATE analysis

Mortality at 30 days

Mortality at 180 Days

Discussion

Limitations

Conclusions

Declarations

References

Abbreviations

Supplementary Files

Status:

Journal Publication

Version 1