External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes identified in the national diabetes program in Iran

doi:10.21203/rs.3.rs-1002732/v1

Download PDF

Research

External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes identified in the national diabetes program in Iran

https://doi.org/10.21203/rs.3.rs-1002732/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

Cardiovascular diseases are the first leading cause of mortality in the world. Practical guidelines recommend an accurate estimation of the risk of these events for effective treatment and care. The UK Prospective Diabetes Study (UKPDS) has a risk engine for predicting CHD risk in patients with type 2 diabetes, but in some countries, it has been shown that the risk of CHD is poorly estimated. Hence, we assessed the external validity of the UKPDS risk engine in patients with type 2 diabetes identified in the national diabetes program in Iran.

Methods

The cohort included 853 patients with type 2diabetes identified between March 21, 2007, and March 20, 2018 in Lorestan province of Iran. Patients were followed for the incidence of CHD. The performance of the models was assessed in terms of discrimination and calibration. Discrimination was examined using the c-statistic and calibration was assessed with the Hosmer–Lemeshow χ2 statistic (HLχ2) test and a calibration plot was depicted to show the predicted risks versus observed ones.

Results

During 7464.5 person-years of follow-up 170 first Coronary heart disease occurred. The median follow-up was 8.6 years. The UKPDS risk engine showed moderate discrimination for CHD (c-statistic was 0.72 for 10-year risk) and the calibration of the UKPDS risk engine was poor (HLχ2=69.9, p<0.001) and overestimated the risk of heart disease.

Conclusion

This study shows that the ability of the UKPDS Risk Engine to discriminate patients who got CHD events from those who did not was moderate and the ability of the risk prediction model to accurately predict the absolute risk of CHD (calibration) was poor and it overestimated the CHD risk. To improve the prediction of CHD in patients with type 2 diabetes, this model should be updated in the Iranian diabetic population.

Endocrinology & Metabolism

Internal Medicine

Cardiovascular disease

coronary heart disease

Risk Score

risk prediction

type 2 diabetes mellitus

United Kingdom Prospective Diabetes Study

Cardiovascular disease (CVD) is the first leading cause of death and one of the most important causes of disease burden in the world [1-5]. Each year, more than 17.5 million people die of cardiovascular disease worldwide, in other words, 31% of all deaths are due to this disease [6-8]. This number is expected to raise to more than 23.6 million by 2030[6-9]. Among diabetics, the incidence of cardiovascular disease is two-four times higher than the general population [10-13]. Practical guidelines and evidence-based medical recommend an accurate estimation of the long-term risk of cardiovascular for effective treatment and care. Estimating the impact of risk factors on the incidence of cardiovascular disease is one of the most important challenges in preventing and controlling these diseases. Several risk assessment models are currently available to estimate CVD risk, some of them, such as systematic coronary risk evaluation (SCORE) and the Framingham risk score model (FRS), have been designed for the general population and underestimate the risk of cardiovascular disease in diabetic people[14-16]. Among several CVD risk assessment models developed in patients with diabetes, the United Kingdom Prospective Diabetes Study (UKPDS) risk engine is the most common model. This model by using risk factors such as age, sex, race, smoking, BMI, blood pressure, total cholesterol, HDL cholesterol and HbA1c estimates the ten-year risk of CVD in patients with diabetes [17]. The results of various studies using the UKPDS risk score to estimate the risk of CVD disease were conflicting. In some societies, the risk of CVD was well-estimated, but in some societies it was overestimation (discrimination of the model was poor or moderate and calibration was poor)[18-23]. This study was designed to investigate the external validation of the UKPDS risk engine in patients with type 2 diabetes identified in the national diabetes program in Iran.

Study Population

In this cohort study among 19453 eligible patients with diabetes registered in the national diabetes program between March 21, 2007, and March 20, 2018 in Lorestan province, 1105 people were randomly selected and their documents reviewed retrospectively. Among these patients, 942 had complete baseline examinations (LDL, HDL, total cholesterol, triglycerides, and HbA1c). but 89 patients had a follow-up period of less than 4 years and were excluded from the study. Inclusion criteria included, newly diagnosed diabetes, as fasting plasma glucose greater than 126mg/dl on two occasions. Exclusion criteria included a history of CHD or stroke before diabetes, impaired endocrine disorder, and severe debilitating disease along with diabetes.

Predictors And Their Measurements

Demographic variables including age, gender, marital status, history of smoking in the lifetime and last year, the type of tobacco used and the date of commencement of smoking, the amount of physical activity during the week, family history of cardiovascular diseases, family history of stroke, and family history of diabetes at the time of diagnosis of diabetes and Seasonally after the diagnosis of diabetes were collected. Also, laboratory tests and clinical examinations Included LDL, HDL, total cholesterol, triglyceride, HbA1c, height, weight, systolic and diastolic blood pressure that were prescribed by a physician at the time of diagnosis of diabetes and once every three months after the diagnosis of diabetes was extracted from patients' medical records. In this study for external validation of the UKPDS risk engine, the variables of age and smoking at the time of diagnosis of diabetes were considered. The variables of HbA1c, systolic blood pressure, and lipid ratio (The ratio of total cholesterol to HDL) were considered the mean of the first and second years. The variable amount of HbA1c using Equation 1 became the international standard (DCCT) format [24]. In addition, total cholesterol, LDL, and HDL variables were converted to mmol/L before entering the model. Table 1 shows the Risk factors included in the CHD model.

Table 1

Risk factors included in the CHD model
Abbreviations	Definitions/values
Age (years)	Age in years at diagnosis of diabetes
Sex	1for female ; 0 for male
Race	1 for Afro-Caribbean ; 0 for Caucasian or Asian-Indian
Smoker (%)	1 for a current smoker, of tobacco in any form, at diagnosis of diabetes ; 0 otherwise
HbA1c( %)	HbA1c( %), mean of values for years 1 and 2
SBP	Systolic blood pressure (mmHg), mean of values for years 1 and 2
LR	Total cholesterol/HDL cholesterol ratio, mean of values for years 1 and 2

Hb(DCCT) =Hb(bio)*0.86) +0.24

Statistical analysis

We excluded individuals with missing values for one or more of the variables (smoking status, total cholesterol, HDL-cholesterol, systolic blood pressure, HbA1c, and ethnicity) used to calculate the UKPDS Risk Engine CHD risk estimates.We calculated the observed CHD risk and the estimated CHD risk using the UKPDS risk engine. The performance of the models was assessed in terms of discrimination (the ability of the risk prediction model to discriminate patients who got a CHD event from those who did not) and calibration (the ability of a risk prediction model to accurately predict the absolute risk of CHD event that is subsequently observed). The discriminative ability of the model was determined by calculating the Harrell's C-index statistic AND Calibration was assessed with the Hosmer–Lemeshow χ2 statistic (HLχ2) test. In our study, a CHD risk equation is considered good if the C-index is ≥0.75. The values between 0.51 and 0.74 are considered moderate and CHD equation with C-index ≤0.5 considered as poor discrimination. All analyses were completed using Stata Version 15.0. (STATA Corp., College Station, Texas, USA). And The confidence level in all analyzes was 95% (error alpha=0.05)

Ethics Statement

This retrospective study was conducted in patients with type 2 diabetes identified in the national diabetes program and approved by the Deputy of Research and Ethics Committee of Iran University of Medical Sciences, Tehran, Iran.

Among 942 new patients with diabetes, 853 had complete data (smoking status, total cholesterol, HDL-cholesterol, systolic blood pressure, HbA1c, and ethnicity) and were included in the study. During 7464.5 person-years of follow-up 170 first Coronary heart disease occurred. The mean and median follow-up were 8.3 and 8.6 years, respectively. In this study, the mean (SD) age at the time of diagnosis of diabetes was 52.6 (11.6) years for women and 55.6 (11.7) years for men. 20% of men and 4% of women smoked, mean (SD) body mass index was 25.2 (4.5) for men and 27 (4.4) for women. Also, the mean first and second year after the diagnosis of diabetes of the variables of hemoglobin A1C, systolic blood pressure, total cholesterol, and HDL cholesterol were 8.5(1.4), 124(16.1), 5.3(0.9), and 1.14(0.3) in men and 8.5(1.4), 123(16.5), 5.2(0.8), and 1.1(0.3) in women, respectively. Table 2 shows the baseline characteristics of the participants in this study and the UKPDS study.In this study, the average age of men was 56 years, which was 4.5 years higher than the UKPDS study but the mean age of women in the two studies was similar. The mean hemoglobin A1C in the subjects was about 2% higher than in the UKPDS. Also, smoking was about 14% for men and 21% for women, and systolic blood pressure was about 10 mm for men and 15 mm in women were less than UKPDS. However, the mean total cholesterol and HDL cholesterol of the two studies were similar.

Table 2

Baseline Characteristics of patients with type 2 diabetes, as mean (S.D.) or percentage (number), in the National Iran Diabetes Register (NIDR )and UKPDS
Variable	UKPDS		NIDR
	Men	women	Men	women
At diagnosis of diabetes
Age (years)	51.5(8.8)	52.7(8.7)	55.6(11.7)	52.6(11.6)
White Caucasian (%)	215 (81)	1603(85)	261(100)	592(100 )
Afro-Caribbean (%)	201 (7.6)	153(8.1)	0(0)	0(0)
Asian-Indian (%	291(11)	141(7.4)	0(0)	0(0)
Smoker (%)	898(34 )	474 (25)	52(19.9)	25(4.2 )
Body-mass index	27.7(4.6)	30.4(6.3)	25.2(4.5)	27.0(4.4)
Mean of values one and two years after diagnosis of diabetes
HbA1c( %)	6.6(1.4)	6.9(1.5)	8.51(1.4)	8.52(1.4)
Systolic blood pressure (mmHg)	133(18)	139(21)	123.6(16.1)	123.1(16.5)
Total cholesterol (mmol/l)	5.2(1.0)	5.7(1.1)	5.25(0.93)	5.19(0.84)
HDL cholesterol (mmol/l)	1.06(0.23)	1.18(0.27)	1.14(0.27)	1.13(0.26)

Table 3

Discrimination and calibration of the UKPDS risk engine for calculated risk periods of 5, 6, 8 and 10 years, with CHD as outcome
Calculated risk period (years)	Discrimination Analysis		Calibration Analysis
Calculated risk period (years)	C-Index	Interpretation	HL χ2	P	Interpretation
5	0.734	Moderate	143.9	0.001	Poor
6	0.728	Moderate	110.5	0.001	Poor
8	0.723	Moderate	75.1	0.001	Poor
10	0.717	Moderate	69.9	0.001	Poor

Performance of the model for CHD outcome

In this study (excluding participants with a shorter follow-up than 4 years), the c-statistic was 0.72 for 10-year risk, in other words, discrimination was Moderate. However, the model calibration was poor (HLχ2=69.9, p<0.001) and overestimated the risk of heart disease. Discrimination and calibration were similar for the 5, 6, and 8-year risk periods. Table 3shows the discrimination and calibration of the risk assessment model for different periods. Figure 1 shows the predicted and observed risk of heart disease in people with type 2 diabetes based on the deciles of the predicted risk. In this study, the predicted risk of heart disease in people with diabetes was 78% higher than the risk of heart disease observed. In other words, the UKPDS risk engine78 percent overestimated the risk of heart disease in patients with type 2 diabetes identified in the national diabetes program in Iran.

This study shows that the ability of the UKPDS Risk Engine to discriminate patients who suffered from CVD events from those who do not was moderate and the ability of a risk prediction model to accurately predict the absolute CVD events (calibration) was poor and overestimated CVD risk prediction. Also, the UKPDS risk engine discrimination for predicting 5 and 8 years CVD events was moderate and calibration was poor and the risk was overestimated.

In a 2006 cohort study by Bo Kyung Koo et al, to evaluate the External validity of the UKPDS Model on 732 Korean diabetics, 46 heart diseases occurred during a 65-month follow-up period. In this study, UKPDS risk engine discrimination was moderate to poor, and the UKPDS risk score overestimated the risk of heart disease in Korean diabetics (AUROC, 0.578 [95% CI, 0.482–0.675])[25].In a study to evaluate the Performance of the UKPDS Cardiovascular Disease Risk score in Germany, 456 patients with diabetes from two population-based studies in southern Germany followed up10 years. The UKPDS risk engine was moderately discriminated (c-statistics =0.64) and overestimated the risk of heart disease[19]. In a study conducted in Malaysia to assess the 10-year risk of cardiovascular disease and compare its estimated risk with the UKPDS cardiovascular risk assessment model, random 660 patients with diabetes were selected to study. Although the UKPDS risk engine estimated the risk of heart disease better than the Framingham model, the UKPDS risk engine had moderate differentiation and poor calibration[26]. In a study conducted in China, 7067 patients with type 2 diabetes were evaluated for heart disease risk. The calibration of the model was tested by the Hosmer-Lemeshow and discrimination by the ROC curve. In this study, 351 new heart diseases occurred during 5.5 years of follow-up. The model discrimination was 0.74%, and the UKPDS risk engine overestimated the risk of heart disease in Chinese diabetics[27]. In a retrospective cohort study conducted in the UK, the external validity of the UKPDS Risk Engine by using routine healthcare data from 79966 patients aged between 35 and 85 years from 1998 to 2011 to assess. The 10-year risk of observed heart disease was 6.1%, and the predicted risk based on the UKPDS risk assessment model was 16.5%, and the UKPDS risk assessment engine was moderately discriminated. As a result, The UKPDS Risk Assessment Engine overestimated the risk of heart disease in UK diabetics[18]. Another study was performed using EPIC-Norfolk cohort study data. 10137 people aged 40 to 79 years were followed up for cardiovascular disease for 10 years, and 69 cases of heart disease occurred. The external validity of the UKPDS and Framingham Risk Engine was assessed، both risk engines overestimated the risk of heart disease[28].In other studies, in different countries, the UKPDS risk engine overestimated the risk of heart disease in diabetics[23, 25, 29].The poor performance of the UKPDS risk engine for predicting CHD was the result of a selection of cohort study data that began in 1977. Diabetes is now detected at an earlier stage, drug treatment begins earlier and therefore reduces the risk of heart disease in the future. Also, the risk factors for heart disease change over time, and some risk factors are better treated.

The main limitation of this study was retrospective cohort design, which resulted in selection bias during excluding the subjects who had insufficient follow-up time to calculate the CHD risk. This study shows that UKPDS risk engine overestimates CHD. To improve the prediction of CHD in patients with type 2 diabetes, this model should be updated in Iranian diabetic patients.

AUROC: Area Under the Receiver Operating Characteristic

BMI: Body mass index

CVD: Cardiovascular Disease

DCCT: Diabetes Control and Complications Trial

DM: Diabetes Mellitus

EPIC-Norfolk: (European Prospective Investigation into Cancer) Norfolk study

FRS: Framingham Risk Score Model

HbA1C: Glycated hemoglobin A

HDL: high-density lipoproteins

LDL: low-density lipoprotein) cholesterol

ROC: Receiver Operating Characteristic

SCORE: Systematic Coronary Risk Evaluation

SD: Standard Devia

UKPDS :UK Prospective Diabetes Study

Ethics approval and consent to participate

The study was approved by the ethics institutional review board of Iran University of Medical Sceinces for studies in the community. It was exempted from signing informed consent forms.

Consent for publication

Not applicable

Availability of data and materials

The data is available on request from the authors

Competing interests

The authors declare that they have no competing interests

Funding

This work was supported by Grants from Iran University of Medical Sceinces ( Nunmber-96-01-27-30511)

Authors' contributions

MV, HRB, DK, SAM: conception or design of the work. MV, DK, HRB data collection, data analysis and interpretation, drafting the article, . MEK, SAM and MSD interpretation and critical revision of the article, final approval of the version to be published. All authors read and approved the final manuscript.

Acknowledgements

We thank all health care professionals in Lorestan province who helped to carry out this study.

Authors' information

Endocrine Research Center, Institute of Endocrinology and Metabolism, Iran University of Medical Sciences, Tehran. Iran

Hamid Reza Baradaran and Mohammad Ebrahim Khamseh

Department of Epidemiology, School of Medicine , Iran University of Medical Sceinces, Tehran, Iran

Mehrdad Valipour Masoud Soleymani Dodaran, Seyed Abbas Motevalian

Prevention of Metabolic Disorders Research Center, Research Institute for Endocrine Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Davood Khalili

1. Countdown, N., 2030: worldwide trends in non-communicable disease mortality and progress towards Sustainable Development Goal target 3.4. Lancet, 2018. 392(10152): p. 1072-88.

2. Shaw, J.E., R.A. Sicree, and P.Z. Zimmet, Global estimates of the prevalence of diabetes for 2010 and 2030. Diabetes research and clinical practice, 2010. 87(1): p. 4-14.

3. Bennett, J.E., et al., NCD Countdown 2030: worldwide trends in non-communicable disease mortality and progress towards Sustainable Development Goal target 3.4. The Lancet, 2018. 392(10152): p. 1072-1088.

4. Roth, G.A., et al., Global, regional, and national burden of cardiovascular diseases for 10 causes, 1990 to 2015. Journal of the American College of Cardiology, 2017. 70(1): p. 1-25.

5. Manemann, S.M., et al., Recent trends in cardiovascular disease deaths: a state specific perspective. BMC Public Health, 2021. 21(1): p. 1-7.

6. Hajar, R., Framingham contribution to cardiovascular disease. Heart views: the official journal of the Gulf Heart Association, 2016. 17(2): p. 78.

7. Al-Mawali, A., Non-communicable diseases: shining a light on cardiovascular disease, Oman’s biggest killer. Oman medical journal, 2015. 30(4): p. 227.

8. Olaniyi, E.O., et al. Neural network diagnosis of heart disease. in 2015 International Conference on Advances in Biomedical Engineering (ICABME). 2015. IEEE.

9. Pyakurel, P., et al., Cardiovascular risk factors among industrial workers: a cross–sectional study from eastern Nepal. Journal of Occupational Medicine and Toxicology, 2016. 11(1): p. 1-7.

10. Yang, W., et al., Comparison between metabolic syndrome and the framingham risk score as predictors of cardiovascular diseases among Kazakhs in Xinjiang. Scientific reports, 2018. 8(1): p. 1-8.

11. Malahfji, M. and J.J. Mahmarian, Imaging to stratify coronary artery disease risk in asymptomatic patients with diabetes. Methodist DeBakey cardiovascular journal, 2018. 14(4): p. 266.

12. Xu, G., et al., Risk of all-cause and CHD mortality in women versus men with type 2 diabetes: a systematic review and meta-analysis. European journal of endocrinology, 2019. 180(4): p. 243-255.

13. Hadaegh, F., et al., New and known type 2 diabetes as coronary heart disease equivalent: results from 7.6 year follow up in a Middle East population. Cardiovascular diabetology, 2010. 9(1): p. 1-8.

14. Lloyd-Jones, D.M., et al., Framingham risk score and prediction of lifetime risk for coronary heart disease. The American journal of cardiology, 2004. 94(1): p. 20-24.

15. Hense, H.-W., et al., Evaluation of a recalibrated systematic coronary risk evaluation cardiovascular risk chart: results from systematic coronary risk evaluation Germany. European Journal of Preventive Cardiology, 2008. 15(4): p. 409-415.

16. Coleman, R.L., et al., Framingham, SCORE, and DECODE risk equations do not provide reliable cardiovascular risk estimates in type 2 diabetes. Diabetes care, 2007. 30(5): p. 1292-1293.

17. Stevens, R.J., et al., The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56). Clinical science, 2001. 101(6): p. 671-679.

18. Bannister, C.A., et al., External validation of the UKPDS risk engine in incident type 2 diabetes: a need for new type 2 diabetes–specific risk equations. Diabetes care, 2014. 37(2): p. 537-545.

19. Laxy, M., et al., Performance of the UKPDS outcomes model 2 for predicting death and cardiovascular events in patients with type 2 diabetes mellitus from a German population-based cohort. Pharmacoeconomics, 2019. 37(12): p. 1485-1494.

20. McEwan, P., et al., Validation of the UKPDS 82 risk equations within the Cardiff Diabetes Model. Cost effectiveness and resource allocation, 2015. 13(1): p. 1-7.

21. Ezenwaka, C., et al., Prediction of 10-year coronary heart disease risk in Caribbean type 2 diabetic patients using the UKPDS risk engine. International journal of cardiology, 2009. 132(3): p. 348-353.

22. Clarke, P., et al., A model to estimate the lifetime health outcomes of patients with type 2 diabetes: the United Kingdom Prospective Diabetes Study (UKPDS) Outcomes Model (UKPDS no. 68). Diabetologia, 2004. 47(10): p. 1747-1759.

23. Van Dieren, S., et al., External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes. Diabetologia, 2011. 54(2): p. 264-270.

24. Marshall SM, B.J., Standardization of HbA1c measurements: a consensus statement. Annals of clinical biochemistry, 2000. 1(1): p. 45-6.

25. Koo, B.K., et al., Prediction of coronary heart disease risk in Korean patients with diabetes mellitus. Journal of Lipid and Atherosclerosis, 2018. 7(2): p. 110-121.

26. Yew, S.Q., Y.C. Chia, and M. Theodorakis, Assessing 10-Year Cardiovascular Disease Risk in Malaysians With Type 2 Diabetes Mellitus: Framingham Cardiovascular Versus United Kingdom Prospective Diabetes Study Equations. Asia Pacific Journal of Public Health, 2019. 31(7): p. 622-632.

27. Yang, X., et al., Development and validation of a total coronary heart disease risk score in type 2 diabetes mellitus. The American journal of cardiology, 2008. 101(5): p. 596-601.

28. Simmons, R.K., et al., Performance of the UK prospective diabetes study risk engine and the Framingham risk equations in estimating cardiovascular disease in the EPIC-Norfolk cohort. Diabetes care, 2009. 32(4): p. 708-713.

29. Piniés, J.A., et al., Development of a prediction model for fatal and non-fatal coronary heart disease and cardiovascular disease in patients with newly diagnosed type 2 diabetes mellitus: the Basque Country Prospective Complications and Mortality Study risk engine (BASCORE). Diabetologia, 2014. 57(11): p. 2324-2333.

Download PDF

Version 1

posted

You are reading this latest preprint version

External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes identified in the national diabetes program in Iran

Status:

Version 1

Abstract

Figures

Background

Methods

Study Population

Predictors And Their Measurements

Statistical analysis

Ethics Statement

Results

Performance of the model for CHD outcome

Discussion

Conclusion

Abbreviations

Declarations

References

Status:

Version 1