The Chinese Coronary Artery Bypass Grafting Registry Study: how well does the EuroSCORE predict operative risk for Chinese population?

Abstract

Objective: The European System for Cardiac Operative Risk Evaluation (EuroSCORE) model is a widely used risk prediction algorithm for cardiac surgery in China due to lack of a local model, although its validation has never been tested. The aim of this study was to assess the performance of the logistic EuroSCORE when applied in the Chinese Coronary Artery Bypass Grafting Registry database, which is representative of adult cardiac surgery in China. Methods: The logistic EuroSCORE model was applied to all patients undergoing coronary artery bypass grafting (CABG) surgery at 35 centres in China between January 2004 and December 2005. The entire cohort and a subgroup of patients undergoing isolated CABG were analysed. The calibration of the EuroSCORE model was assessed by comparing observed and predicted in-hospital mortalities. Discrimination was tested by determining the area under the receiver operating characteristic (ROC) curve. Results: The Chinese CABG Registry Study recruited 9248 patients. There were significant differences in the prevalence of risk factors between the Chinese population and European cardiac surgical populations. For the entire cohort, the predicted mortality was 5.51% and observed mortality was 3.27%. Of the isolated CABG subset of 8120 patients, the predicted mortality was 4.21% and the observed mortality was 2.22%. The logistic EuroSCORE overpredicted observed mortality for the entire cohort and the isolated CABG subset at all risk tertiles to different degrees. The discriminative power of EuroSCORE was acceptable but not very satisfactory, with an area under the ROC curve of 0.72 for the entire cohort and 0.71 for the isolated CABG subset. Conclusion: EuroSCORE model does not accurately predict outcomes in this group of Chinese CABG patients. Therefore, caution should be exercised when using it for risk prediction in China. Creation of a new model, which accurately predicts outcomes in Chinese CABG patients, is warranted.

1 Introduction

Risk prediction in current cardiac surgical practice plays an important role in surgical decision-making, preoperative patient education and consent, and quality assurance measures. It also enables benchmarking and meaningful comparisons between centres. With no knowledge of the current risk profile of Chinese cardiac surgical patients, crude mortality rate in a centre was once used as a risk indicator in China although it is usually not related to preoperative risk factors. The European System for Cardiac Operative Risk Evaluation (EuroSCORE) [1,2], aiming to predict 30-day mortality of the European cardiac surgical patients, was introduced to China for risk-adjustment in the year of 2000 due to the absence of a local risk prediction model. More recently the logistic EuroSCORE has been used, which is claimed to be a better predictor of operative risk than the additive model [3,4]. Now the EuroSCORE model is the most widely used risk prediction algorithm in China [5] owing to its validation in European, North American and Japanese populations [4,6–10]. However, validation of the EuroSCORE model has never been tested in the patient population that was representative of contemporary cardiac surgery in China. Hence, there existed doubt as to whether or not the EuroSCORE model for risk prediction was appropriate in China [5,11].

Over the last decade patients undergoing coronary artery bypass grafting (CABG) have been increasing rapidly in China due to a higher incidence of coronary artery disease in the developing world [12]. Meanwhile, they constitute a major and challenging group of patients for adult cardiac surgery. Fuwai Hospital, Beijing, China has set up a national multi-centre database of patients undergoing CABG in 2006. This database, named the Chinese Coronary Artery Bypass Grafting Registry Study [13], is now without doubt the largest of its kind in China. We have therefore analysed the predictive ability of the EuroSCORE on this large contemporary coronary artery surgery database in China, aiming to assess the EuroSCORE model within the Chinese Coronary Artery Bypass Grafting Registry Study. Hence, two specific questions were to be addressed: Primarily, is the EuroSCORE a good overall operative risk predictor for coronary artery surgery in China? Secondly, does the EuroSCORE predict observed mortality well in low, medium and high-risk subgroups?

2 Methods

The Chinese CABG Registry is a national multi-centre study for the primary purpose of risk stratification and outcome assessment following CABG in Chinese adults as well as providing a potential clinical research tool for the future. Its steering group was set up to include a number of cardiac surgeons and epidemiologists from Fuwai Hospital. The registry recruited 9248 consecutive CABG patients from January 2004 to December 2005, and contained detailed information on patient demographics, preoperative risk factors, operative details, postoperative hospital course and morbidity and mortality outcomes. In our study mortality was defined as any in-hospital death.

The Chinese and European patient populations were compared for demographic characteristics, incidence of surgical procedures performed and prevalence of risk factors. Nevertheless, the definitions of some of the risk factors were not identical to the EuroSCORE definition. However, a close approximation was achieved. The risk factors, together with their corresponding definitions for comparison, are listed in Table 1 . The logistic EuroSCORE model was then tested on two groups of patients: the entire cohort population (n = 9248), and isolated CABG population (n = 8120). As one of the criticisms of the additive EuroSCORE is that it does not predict well in higher-risk patients [9], we have studied the performance of the logistic EuroSCORE in low-, medium- and high-risk patients by dividing the groups into three subgroups according to the EuroSCORE.

Statistical analyses were performed using SPSS version 13.0 (SPSS Inc., Chicago, Illinois). Continuous variables were presented as mean ± SD and comparisons between them were performed using Student’s t test, whereas categorical variables were shown as a percentage and differences in them were assessed using Fisher’s exact test or χ² test. A p value less than 0.05 was considered significant.

Performance of the EuroSCORE model was assessed by comparing the observed and predicted mortality figures. We have simply compared predicted with observed mortality to check correct calibration of the EuroSCORE. To assess the model discrimination (statistical accuracy) we have used the area under the receiver operating characteristics (ROC) curve [14]. An area of 0.5 reflects no discrimination and an area of 1.0 indicates a perfect predictor. Areas of greater theater than 0.7 are generally thought to be useful.

3 Results

In all, 35 centres from 15 provinces in China participated in the project, giving a total of 9290 patients undergoing CABG surgery during the study period. Following the error checking procedures, 42 patients were eliminated from the study, leaving 9248 patients for analysis. Table 2 shows the breakdown by operative category.

3.1 Demographics

There were significant differences between the Chinese and European surgical populations. The prevalence of risk factors in the two populations is shown in Table 3 . Chinese population was more likely to have or be labeled as having hypertension, diabetes, cerebrovascular accident, unstable angina, triple-vessel disease, left main disease, pulmonary hypertension and emergency operation. European population had proportionately more females, and more European patients had or were labeled as having chronic pulmonary disease, extracardiac arteriopathy, previous cardiac surgery, left ventricular dysfunction, other than isolated CABG and surgery on thoracic aorta. All differences were highly significant (p ≪ 0.001). The similarities between the two populations were age, critical preoperative state, recent myocardial infarction and postinfarct septal infarction.

3.2 Calibration and discrimination

Table 4 shows the predictive ability of the logistic EuroSCORE. Of the entire cohort of 9248 patients, there were 302 deaths observed, giving an overall observed mortality rate of 3.27%. The EuroSCORE model predicted a mortality rate of 5.51%, meaning that the logistic EuroSCORE needs to be calibrated by a factor of 0.59 to give an accurate representation of operative risk for Chinese population. Of the isolated CABG subset of 8120 patients, there were 180 deaths observed, giving an observed mortality rate of 2.22%. The EuroSCORE model predicted a mortality rate of 4.21%, meaning that the logistic EuroSCORE needs to be calibrated by a factor of 0.53. The discriminatory ability of logistic EuroSCORE performance was acceptable but not very satisfactory, with an area under the ROC curve of 0.72 for the entire cohort and 0.71 for the isolated CABG subset.

Table 5 shows the predictive ability of the logistic EuroSCORE in low-, medium- and high-risk patients. Again the discriminatory ability of EuroSCORE was not satisfactory for the entire cohort and the isolated CABG subset, with the area under the ROC curve ranging from 0.58 to 0.71. For the entire cohort the logistic EuroSCORE overpredicted observed mortality at each risk tertile. It did so by a similar degree for medium- and high-risk patents but by a lesser degree for low-risk patients. These findings were similar for the isolated CABG subset.

4 Discussion

4.1 Principal findings

We have shown, in this study of 9248 patients from 35 Chinese cardiac surgical centres, that the logistic EuroSCORE does not accurately predict outcomes in this group of Chinese CABG patients. This is true of a mixed group of the entire cohort as well as those undergoing isolated CABG. The logistic EuroSCORE model overestimates observed mortality to different degrees at all risk tertiles. Furthermore, the discriminatory ability of the models was not very satisfactory. Thus, the EuroSCORE should be used with caution for risk adjustment or risk prediction in China.

4.2 Meanings and discussions of our findings

Analyses on measuring mortality outcomes with risk prediction models in cardiac surgery of hospitals and surgeons are increasing, and risk prediction models play a more important role in current cardiac surgical practice. Furthermore, it has the advantage of allowing meaningful comparisons to be made regionally and internationally. The use of a ready-made widely used model seems to be an optimal choice available to a cardiac surgical centre. However, a ready-made model may not accurately predict local outcomes and require validation prior to the use of it [11].

That the EuroSCORE model, which was developed and introduced in the late 1990s, overpredicts mortality in contemporary Chinese patients is of no surprise. However, the reasons are likely to be multi-factorial. One possible reason is the improvement in medical care and good performance of the heart centres involved. The past decade heralded advances in surgical, anaesthetic, perfusion procedures and postoperative intensive care. This progress led to overall reductions in surgical mortality. The STS database from United States demonstrates that despite higher risk, observed operative mortality rate for isolated CABG decreased from 3.9% in 1990 to 3.0% in 1999 [15]. Similar observations have been made using databases from other countries. The observed mortality rate for isolated CABG in the database from the north west of England was 2.0% between April 2002 and March 2004 [16]. In an analysis of Australian Society of Cardiac and Thoracic Surgeons (ASCTS) Cardiac Surgery Database, the observed operative mortality for isolated CABG was also 2.0% between July 2001 and July 2005 [17]. In our study the observed mortality rate for isolated CABG was 2.22%, suggesting good performance of heart centres in contemporary China.

Another possible reason is that the epidemiology of coronary heart disease and comorbidities in China may be different to those of the European population. However, it is likely to be unreal due to the difference in access to health care and management of these comorbidities between China and Europe. The higher prevalence of chronic pulmonary disease, extracardiac arteriopathy and previous cardiac surgery in European patients may be just a reflection of better access to medical care in a highly government-subsidised medical system. However, we are unable to know whether the failure of the EuroSCORE model in our study is due to factors unique to China or factors common to cardiac surgery worldwide.

4.3 Strengths and weakness of the study

We have assessed the predictive ability of the EuroSCORE on a large contemporary coronary surgical database of Chinese population, to which 35 centres contributed. The database was subjected to an internal audit and quality assurance measures and therefore we believe that the data accuracy is high and that our findings are indicative of the national picture. However, the data have only looked at coronary artery surgery rather than at other highly specialised areas such as aortic surgery or valvular surgery. Hence, the validation of EuroSCORE in these groups remains unknown. Furthermore, our data have not been validated externally, which is also a weakness of our study.

4.4 Future research

Our findings also suggest directions in which future research needs to go. Firstly, the EuroSCORE was developed on data from patients in 1995 and may not reflect contemporary cardiac surgical practice in Europe. Hence, there is a real need for a European multi-centre revalidation of the EuroSCORE to establish its validity in current cardiac surgical practice. Only then can we know whether the failure of the model in our study is likely to be due to factors unique to a region or factors common to cardiac surgery worldwide. Secondly, if a ready-made model does not accurately predict outcomes regionally, the choices available to a region or a heart centre are to recalibrate the existing model or creating a new model [11]. However, both options are impractical to an individual heart centre as they require a comprehensive database. Creation is certainly superior to recalibration if the existing model does not have a very good discriminatory ability. Therefore, creation of a new model on our multi-centre database is the direction of our further study.

5 Conclusion

We have shown that the EuroSCORE model does not accurately predict outcomes in this group of Chinese coronary artery surgical patients. Therefore, caution should be exercised when using it for risk prediction in China. Creation of a new model, which accurately predicts outcomes in Chinese coronary artery surgical patients, is warranted.

Appendix A The 35 centres participating in the Chinese CABG registry study

Cardiovascular Institute and Fuwai Hospital, Nanjing Medical University First Hospital, Shanxi Provincial Heart Hospital, Shanghai First People’s Hospital, Shandong Liaocheng People’s Hospital, Beijing Chaoyang Hospital, Henan Chest Hospital, Henan Provincial People’s Hospital, Qingdao Municipal Hospital, Wuhan Asia Heart Hospital, Nanjing Southeast University Zhongda Hospital, Anyang People’s Hospital, Jinan PLA Hospital, Xuzhou Central Hospital, Changhai Hospital, Qiangdao University Hospital, Xijing PLA Hospital, Guangdong Provincial Cardiovascular Institute, Fujian Medical University Union Hospital, Shenyang PLA Hospital, Fudan University Zhongshan Hospital, Shanghai Chest Hospital, Hebei Medical University Second Hospital, Beijing University First Hospital, Beijing University People’s Hospital, Beijing University Third Hospital, Sino-Japan Friendship Hospital, Shanghai Second Medical University Ruijin Hospital, Tianjin Chest Hospital, China PLA General Hospital, Fujian Provincial Cardiovascular Institute, Tongji Medical University Union Hospital, China Medical University First Hospital, Shangdong University Qilu Hospital, and Qingdao Fuwai Hospital.

Acknowledgements

This study was conducted on behalf of the Chinese CABG Registry Study. The consultant surgeons involved are Shengshou Hu, Feng Xiao, Feng Wan, Bibo Yang, Peixiong Su, Fenglin Wang, Zhigang Guo, Su Liu, Shengjun Ma, Junsheng Wang, Shunye Zhang, Jiali Liang, Zhaoyun Cheng, Huishan Wang, Kaiguang Zhang, Yifan Chi, Ping Xu, Xin Chen, Zhiyong Liu, Biao Yuan, Dinghua Yi, Liang Tao, Mingdi Xiao, Zhiyun Xu, Qiang Zhao, Yingze Li, Ye Kong, Ruobin Wu, Haisheng Chen, Tao Han, Liangwan Chen. We acknowledge the assistance of each centre for collecting and validating the data. We also thank Dr Lixin Jiang and the support of Fuwai-Oxford Collaborative Research Centre for their hard work and tireless efforts.

References

Author notes