Background: The Pooled Cohort Equations (PCEs) are race- and sex-specific Cox PH-based models used for 10-year atherosclerotic cardiovascular disease (ASCVD) risk prediction with acceptable discrimination. In recent years, neural network models have gained increasing popularity with their success in image recognition and text classification. Various survival neural network models have been proposed by combining survival analysis and neural network architecture to take advantage of the strengths from both. However, the performance of these survival neural network models compared to each other and to PCEs in ASCVD prediction is unknown.
Methods: In this study, we used 6 cohorts from the Lifetime Risk Pooling Project and compared the performance of the PCEs in 10-year ASCVD risk prediction with an all two-way interactions Cox PH model (Cox PH-TWI) and three state-of-the-art neural network survival models including Nnet-survival, Deepsurv, and Cox-nnet. For all the models, we used the same 7 covariates as used in the PCEs. We fitted each of the aforementioned models in white females, white males, black females, and black males, respectively. We evaluated models’ internal and external discrimination power and calibration.
Results: The training/internal validation sample comprised 23246 individuals. The average age at baseline was 57.8 years old (SD = 9.6); 16% developed ASCVD during average follow-up of 10.50 (SD = 3.02) years. Based on 10x10 cross-validation, the method that had the highest C-statistics was Cox PH-TWI (0.7372) for white males, PCE (0.7973) for white females, Cox PH-TWI (0.6989) for black males, and Deepsurv (0.7874) for black females. In the external validation dataset, PCE (0.7102), Deepsurv (0.7293), PCE (0.6907), and Nnet-survival (0.7243) had the highest C-statistics for white male, white female, black male, and black female population, respectively. Calibration plots showed that in 10x10 validation, PCE had good calibration in white male, white female, black male but was outperformed by Deepsurv in black female. In external validation, all models overestimated the risk for 10-year ASCVD except for Deepsurv in black female.
Conclusions
We demonstrated the use of the state-of-the-art neural network survival models in ASCVD risk prediction. Neural network survival models and PCEs have generally comparable discrimination and calibration.