Is the new EuroSCORE II a better predictor for transapical aortic valve implantation?^†

Abstract

OBJECTIVES

Conventional surgical risk scores are used to identify suitable candidates for transapical aortic valve implantation (TA-AVI) at present. However, these scores do not consider multiple high-risk conditions, including porcelain aorta, mediastinal irradiation or frailty. The aim of this study was to compare the predictive ability of the new EuroSCORE II with the surgical risk scores currently in use.

METHODS

From February 2006 to May 2011, 360 consecutive high-risk patients, age 81.6 ± 6.4 years, 64.4% female, were included using the Edwards SAPIEN™ prosthesis. The prognostic value of the EuroSCORE II was evaluated and compared with the logistic EuroSCORE and STS mortality score by receiver operating characteristics (ROC) curve analysis. In addition, a Spearman correlation analysis was performed, and a stepwise multivariate Cox regression used to identify the independent risk factors of mortality.

RESULTS

The STS score and EuroSCORE II (r = 0.504, P < 0.001) showed a good correlation, while a strong correlation was found between the logistic EuroSCORE and EuroSCORE II (r = 0.717, P < 0.001). Thirty-day and in-hospital mortality rates were 10.6% (38 of 360) and 11.4% (41 of 360), respectively. In-hospital mortality rate was estimated by the logistic EuroSCORE: 30.0 ± 15.7%, the STS score: 11.7 ± 7.8% and the EuroSCORE II: 6.7 ± 5.1%. The prognostic values of the STS score, logistic EuroSCORE and the recent EuroSCORE II systems were analysed by ROC curve analysis for the prediction of 30-day (area under the curve, AUC: 0.64 vs 0.55 vs 0.50) and in-hospital mortality (AUC: 0.65 vs 0.54 vs 0.49). Multivariate regression analysis revealed length of preoperative hospital stay >5 days, body weight <65 kg, preoperative aortic annular diameter ≤20 mm, vital capacity <70% and concomitant mitral regurgitation >1+ as independent risk factors.

CONCLUSION

In patients undergoing TA-AVI, the new EuroSCORE II correlates strongly with the logistic EuroSCORE, but is a poorer predictor of 30-day and in-hospital mortality than the STS score. A true transcatheter aortic valve implantation risk score would be desirable beyond the established scores.

INTRODUCTION

Conventional surgical risk scores are used to identify suitable candidates for transapical aortic valve implantation (TA-AVI). At present, the two most commonly used risk scores are the European System for Cardiac Operative Risk Evaluation (EuroSCORE) and the Society of Thoracic Surgeons Predicted Risk of Mortality (STS-PROM). Both were developed to assess the mortality risk for cardiac surgical procedures.

Whereas the EuroSCORE was initially conducted in 1995 and first published in 1999, the STS score was initially developed in the late 1980s and its current model for solely isolated aortic valve replacement (AVR) was introduced in 2007 [1–3]. The EuroSCORE is a risk-stratification system based on a large European database including 14 871 patients predominately derived from a population undergoing coronary artery bypass graft surgery (CABG). In contrast, the STS score uses different risk models for isolated CABG, isolated valve procedures (with 67 292 patients for isolated AVR) and isolated valve procedures plus CABG. Both risk scores define operative mortality as mortality within 30 days from operation or later if the patient is still hospitalized [4].

As adult cardiac surgery has gone through major changes in terms of surgical techniques and improved perioperative care, risk-adjusted mortality has fallen by around a half in comparison with the early 1990s [5, 6]. In addition, the changing risk profile of cardiac surgical patients over the last decade led the EuroSCORE investigators to develop a revised version, the EuroSCORE II [6].

Therefore, the aim of this study was to compare the predictive ability and properties, as well as the correlation of the new EuroSCORE II with the surgical risk scores currently in use prior to TA-AVI.

METHODS

Study design

After approval by the local ethics committee of the University of Leipzig, and written consent obtained from each patient, 360 consecutive high-risk patients were included between February 2006 and May 2011 in this single-centre study.

Primary end-points of this single-arm study were the predictive ability of the new EuroSCORE II compared with the logistic EuroSCORE and STS mortality score by ROC curve analysis. Secondary end-points were their properties for the definition of operative mortality and correlations between the surgical risk scores. In addition, multivariate Cox regression was used to identify potential independent risk factors for mortality.

Clinical inclusion criteria were age ≥75 years, NYHA functional class II or higher, written informed consent and comorbidities leading to a logistic EuroSCORE ≥15%. By standard protocol, all patients underwent transthoracic and transoesophageal echocardiography. Echocardiographic inclusion criteria were severe degenerative aortic valve stenosis indicated by an aortic valve area ≤1.0 cm² and/or a jet velocity >4 m/s and/or a mean gradient >40 mmHg.

Risk factors recorded in our institutional registry were categorized according to the original EuroSCORE criteria. The EuroSCORE II introduced some new variables, which were additionally collected in our registry. The variable ‘poor mobility’ was defined as when the patient was dependent on a walking frame or rollator.

Transcatheter aortic valve

All treatment options including conventional surgery were discussed in an interdisciplinary Heart Team conference. All patients received the Edwards SAPIEN™ Transcatheter Heart Valve or SAPIEN XT™ composed of a pericardial xenograft fixed within a stainless steel or cobalt chrome, balloon expandable stent (Table 2, Edwards Lifesciences, Irvine, CA, USA). Patients with an aortic annulus diameter of 19–22 mm received a 23-mm Edwards SAPIEN™ and those with an aortic annulus diameter between 22 and 24 mm received a 26-mm prosthesis. The 29-mm Edwards SAPIEN™ has been officially available since January 2011 and used for patients with an aortic annulus diameter between 24 and 27 mm.

Statistical analysis

For statistical analysis, data were 100% complete. All statistical analyses were performed using SPSS, version 16.0 (Chicago, IL, USA) and the R-statistical package (Auckland, New Zealand, version 2.15.0). Continuous variables were analysed by two-tailed Student's t-test or by Mann–Whitney U-test for non-normally distributed variables. The prognostic value of the surgical risk scores was assessed using a receiver operating characteristics (ROC) analysis, producing an area under the curve (AUC) with 95% confidence intervals (CIs). Calibration of these risk-scoring methods was tested by the Hosmer–Lemeshow statistics. The AUC of each surgical risk score was compared according to the method of DeLong et al. [7]. A EuroSCORE II cut-off value of 5% was chosen, since there was no significant difference in terms of operative mortality for any cut-off value. Nevertheless, 5% provided the best expected-to-observed mortality ratio (0.84) and the best discriminatory power in terms of sensitivity and specifity.

Univariate and stepwise multivariate Cox regression analysis of predictors of 30-day mortality were examined (see Supplementary Material). Those variables that had a probability value of ≤0.1 by univariate analysis were included in the multivariate model, and corresponding hazard ratios and 95% CIs were determined. A Spearman correlation analysis was performed to assess the association between the logistic EuroSCORE, EuroSCORE II and STS mortality score. A P-value of ≤0.05 was considered statistically significant. Continuous variables are expressed as mean ± standard deviation for Gaussian distributed variables and otherwise, median values (interquartile range). Categorical data are given as proportions.

RESULTS

Preoperative characteristics of the patients are presented in Table 1. Mean age was 81.6 ± 6.4 years and 64.4% were female. Thirty-day and in-hospital mortality rates were 10.6% (38/360) and 11.4% (41/360), respectively. In-hospital mortality rate was estimated by the logistic EuroSCORE: 30.0 ± 15.7%, the STS score: 11.7 ± 7.8%, and the EuroSCORE II: 6.7 ± 5.1%. Patients demonstrated a consistently high-risk profile throughout the study.

Intraprocedural device success was 95% (342/360) according to the recently updated VARC-2 criteria. The majority of the patients were treated off-pump (91.1%) with 9 requiring conversion to open AVR. Perioperative parameters are shown in Table 2.

With regard to the postoperative outcome, periprocedural stroke/transient ischemic attack (TIA) occurred in only 1.9% and a further 2.2% of patients had a spontaneous stroke during their hospital stay (minor: 1.1% and major: 3.1%, Table 3). Other adverse events consisted of major vascular complications in 3.4% of patients, life-threatening or disabling bleeding in 6.7% and acute kidney injury (modified AKIN classification stage 1: 8.9%, stage 2: 3.1% and stage 3: 16.7%).

Regarding the correlation analysis, the logistic EuroSCORE and the EuroSCORE II showed a moderate correlation with the STS score (r = 0.422, P < 0.001 and r = 0.504, P < 0.001). A strong correlation was found between the logistic EuroSCORE and the EuroSCORE II (r = 0.717, P < 0.001).

The prognostic value of the STS score, logistic EuroSCORE and the recent EuroSCORE II systems was analysed in an ROC curve analysis for the prediction of 30-day (AUC: 0.64 vs 0.55 vs 0.50, Figure 1) and in-hospital mortality (AUC: 0.65 vs 0.54 vs 0.49, Figure 2). The STS score had a better predictive ability than the new EuroSCORE II for 30-day and in-hospital mortality (P = 0.05 and P = 0.01, Table 4).

According to an STS cut-off value of 10%, as proposed by the existing guidelines [8], 161 patients were identified as high-risk patients for conventional surgery, with 26 deaths during the in-hospital stay (16.1%). 240 and 177 patients were identified as surgical high risk according to the cut-off value of 20% for the logistic EuroSCORE and 5% for the EuroSCORE II, with 32 and 21 in-hospital deaths (13.3 and 11.9%, Table 5). However, for the STS score, 13 patients, for the logistic EuroSCORE 8 patients and for the EuroSCORE II, 19 patients who actually suffered operative mortality were below the identified cut-off values. These patients had a mean (standard deviation) STS score of 6.7 ± 1.4, a logistic EuroSCORE of 13.8 ± 5.5 and a EuroSCORE II of 3.3 ± 1.0. Therefore, all three risk models did not identify them as high-risk patients.

Of all preoperative and intraoperative factors analysed (see Supplementary Material), length of preoperative hospital stay >5 days, body weight <65 kg, preoperative annulus diameter ≤20 mm, a reduced vital capacity of <70% and concomitant mitral regurgitation of >1+ could be identified as the only independent risk factors for 30-day mortality (Table 6). In contrast, the most commonly used risk scores (STS score >10%, logistic EuroSCORE >20%) again failed to predict 30-day mortality by stepwise multivariate regression.

DISCUSSION

Advances in surgical techniques and perioperative care have steadily reduced procedural risk of AVR to low levels, even in patients at high surgical risk. However, severe or even fatal complications are somewhat unpredictable and unavoidable. Until now, the two most common risk models used for transcatheter aortic valve implantation (TAVI) are the EuroSCORE and the STS-PROM. In as much as both models were based on patients who have actually undergone surgical AVR, their accuracy in selected high-risk patients is necessarily speculative. Still, they are the best tools we currently have to help select patients and assess outcomes [9].

In this retrospective, single-centre analysis, the STS, the logistic EuroSCORE as well as the EuroSCORE II, did not demonstrate an optimal ability to predict 30-day and in-hospital mortality. The ROC curve analysis demonstrated that the STS score was superior compared to the EuroSCORE II. In very high-risk patients, the EuroSCORE II may actually underestimate 30-day and in-hospital mortality risks.

As the logistic EuroSCORE has been clearly demonstrated to over-predict expected mortality, particularly during valvular heart surgery [9, 10], it may not be the ideal risk score for selecting patients referred to TAVI [11, 12]. This over-prediction, however, should not be surprising, as the EuroSCORE was predominately derived from a population of patients undergoing CABG [13]. In transapical cases, the STS score seems to be a better tool to assess the ‘true’ risk; nevertheless, important factors that are currently only assessable by the so-called eyeball-test are missing. The use of different STS-models for isolated procedures may explain the higher AUC and the potential weakness of a ‘one-model-fits-all’ concept [14].

Finally, the revised EuroSCORE II has been minimally improved with some additional variables, but considerable variability remains excluded [15]. Potential reasons for the low predictive capacity of the EuroSCORE II for TA-AVI patients can be summarized as follows:

• The present EuroSCORE was elaborated using a mixed patient population undergoing cardiac surgery with the majority treated for coronary artery disease, and only a minority for primary isolated AVR.

• Predicted mortality has progressively become less well calibrated to absolute risk across time, with even greater impact at the extremes of risk. Nevertheless, it still accurately stratifies risk because it retains a strong association with mortality.

• The prevalence of risk factors in patients referred for heart surgery may also change over time.

• Only the most common and prevalent risk factors for 30-day mortality are incorporated in the EuroSCORE risk model. Rare risk factors and unusual combinations seem to be more common at the extremes of risk and might eventually lead to greater mortality than predicted.

Although the STS score outperformed the logistic EuroSCORE, both the logistic EuroSCORE and the STS score revealed suboptimal discriminatory power and calibration. This may not be a surprise when extrapolating a surgical risk score to the outcomes of a particular form of therapy such as TA-AVI. Piazza et al. [16] recently compared the performance of these two risk scores in TAVI patients. As in our study, the STS score had a better predictive ability than the logistic EuroSCORE, but neither had optimal prediction of 30-day mortality [16]. Comparable results were found by Wendt et al. [11], Piazza et al. [16] and Ben-Dor et al. [17].

Consistent with this report, Chalmers et al. [14] documented similar results regarding the EuroSCORE II in 814 patients undergoing isolated AVR.

With regard to the correlation analysis, we found a moderate correlation between the logistic EuroSCORE and STS score. Recent studies including Ben-Dor et al. [17] and Piazza et al. [16] reported a moderate linear relationship between the logistic EuroSCORE and STS score (r = 0.61, P < 0.001 and r = 0.58, P < 0.001, Pearson correlation coefficient). In addition, we found a strong correlation between the logistic EuroSCORE and EuroSCORE II.

In the multivariate analysis, independent predictors of 30-day mortality were length of preoperative hospital stay >5 days, body weight <65 kg, preoperative annulus diameter ≤20 mm, a reduced vital capacity of <70% and concomitant mitral regurgitation of >1+.

An increased length of preoperative hospital stay might be a surrogate for multiple comorbidities or a complex course from the beginning, but it seems interesting as this risk factor was not investigated in earlier studies.

Several studies confirmed that low body weight was associated with increased mortality, whereas higher weight was not [18, 19]. In addition, an aortic annulus diameter ≤20 mm, identified as independent risk factors for 30-day mortality, is consistent with this finding. The surgical management of elderly patients with severe aortic stenosis and a small aortic annulus can sometimes be quite challenging. Thus, this specific patient subgroup represents an important area in need of further clinical investigation.

Poor respiratory function, defined as vital capacity <70%, and mitral regurgitation >1+ were reconfirmed as independent risk factors as shown in some earlier series [20, 21]. The most commonly used risk scores (STS score >10%, logistic EuroSCORE >20%) failed to predict mortality.

It was interesting to note that carotid artery disease of >70% was associated with a higher 30-day mortality. In contrast, in a study by Wendler et al. [22], carotid artery stenosis was found to be one of the only two multivariate predictors for 30-day mortality, but in a counterintuitive direction. However, their results might be due to a statistical phenomenon only (false reference category).

The performance of scoring systems in high surgical-risk patients is still important, as it is sometimes difficult to choose the appropriate treatment option in such patients. Even in the absence of a high preoperative surgical risk, many ‘extreme’ or rare conditions (porcelain aorta, frailty, poor mobility and nutrition status, cirrhosis, chest wall deformity, highly compromised respiratory function, previous chest radiation etc.) will justify a TAVI procedure [8]. Selection of candidates for TA-AVI especially risk assessment, should always involve a multidisciplinary team approach including anaesthetists, cardiologists and cardiac surgeons [17].

A true TAVI risk score should be mainly based on risk factors, as earlier stated [21–23] and in part newly identified here, to achieve a better accuracy in TAVI risk prediction in the future. Finally, it should be noted that the STS/ACC transcatheter valve therapy registry plans to develop new models based exclusively on a TAVI population in order to tailor predictions to this carefully selected patient subgroup.

LIMITATIONS

The current study has several limitations, most importantly the retrospective and the single-centre design of the study, although data were collected prospectively. Further limitations are the definition of ‘operative mortality’ limited to a 30-day observation. Future studies should extend the observation time for operative-related mortality, and may include better indicators of quality of life such as severe disabling conditions [24]. Currently available risk scores have obvious limitations, because patients can be at a very high operative risk, yet have low surgical scores [17].

CONCLUSION

To shed light on the performance of the new EuroSCORE II, we present here a single-centre analysis in 360 transapical patients. In patients undergoing TA-AVI, the new EuroSCORE II correlates strongly with the logistic EuroSCORE, but is a poorer predictor of 30-day and in-hospital mortality than the STS score. The new EuroSCORE II may actually underestimate 30-day and in-hospital mortality risks in high-risk patients. A true TAVI risk score would be desirable beyond the established scores.

SUPPLEMENTARY MATERIAL

Supplementary material is available at EJCTS online.

Conflict of interest: Commercial Relationship: Michael Andrew Borger: Speakers Bureau/Honoraria, Edwards Lifesciences LLC.

ACKNOWLEDGEMENTS

We acknowledge the statistical support from Meinhard Mende, senior biometrician at the Coordinating Centre for Clinical Trials at the University of Leipzig and Cornelia Oette and Klaus Krämer (both perfusionists at the Heart Center Leipzig) for transcatheter valve crimping throughout this study.

REFERENCES

APPENDIX. CONFERENCE DISCUSSION

Dr J. Obadia(Lyon, France): This presentation reports the experience of the Leipzig team, which has been a pioneer in TAVI procedures, with a considerable number of patients leading to a very interesting analysis of the prognostic value of the three most commonly used risk scores, the two EuroSCOREs and the STS score.

Your first conclusion is that there is a good correlation between the two EuroSCOREs, and this is not really surprising since the two EuroSCOREs share the same base. Moreover, it is already well known that the logistic EuroSCORE at least doubles the real risk of mortality and the EuroSCORE II underestimates the risk, which your presentation also confirmed. Your second conclusion is that the STS score is less correlated to the two EuroSCOREs and that it gives a better prediction of mortality after TAVI procedures, which is also not a surprise since the STS score was designed for valve surgery, unlike the original EuroSCORE. Consequently, we could say that your paper is simply a confirmation of a well-known reality, the poor quality of the current scores. Nevertheless, the main point of your paper is to put accurate numbers on this reality using a sophisticated statistical analysis based on 100% data collection. This leads me to three questions.

First question. Your paper analysed 360 high-risk patients from the early phase of this innovation. Actually, you report 9% of this group with an implantation performed on bypass, which corresponds to the very beginning of this experiment. Don't you think that this heterogeneity could have impacted the results? Did you test the early phase and the more recent period when a less disappointing prediction of the scores could be expected? Maybe you can answer the first question and then I will ask the other ones.

Dr Haensig: It is certainly difficult to answer this question and I totally agree with you. However, the point I want to make is that “high-risk” might be based on heterogeneity, “the edge of the stars”, as Professor Paul Sergeant stated earlier this morning in the live TV discussion session. If you take out every factor which could have influenced your results, you might get an ideal patient population. However, you will finally end up taking out all the risk factors that define your patient as “high-risk”. Nevertheless, regarding the more recent period, there was a slightly better performance for the logistic EuroSCORE and EuroSCORE II; however, the STS still outperformed both scores.

Dr Obadia: This is my second question. When you say that the positive predictive value of the two EuroSCOREs is between 10 and 13%, which is very low, would you recommend definitely stopping using the EuroSCOREs for the TAVI procedure in the future?

Dr Haensig: I think it is an important message that the STS score has been shown to outperform the logistic EuroSCORE. Since your question is heading towards the existing and maybe even the upcoming TAVI guidelines, coming up with these data, I currently just can recommend using the STS score.

Dr Obadia: And my last question. The positive predictive value of the STS score of 16%, while slightly better, is not very accurate, so that even the STS score is not reliable. And we agree with your wish for a specific score for TAVI. This is very sensitive since surgery is in competition with TAVI and today there is a general agreement to reserve TAVI procedures for the sickest patients. As a result, it is absolutely mandatory to get a more precise tool to define the high risk patients today. Since none of the current scores answer this need, could you define what should be the main items involved in the future ideal TAVI score?

Dr Haensig: I am truly convinced that only a sufficiently large TAVI database will show real-world significant risk factors for this specific patient population and also bring these weights into the correct measure. We will probably not get too many new risk factors into it. However, many extreme or rare conditions such as porcelain aorta, frailty, or highly compromised respiratory function will clearly be in the competition. But I am sure that only a large enough TAVI registry will show if they will truly contribute to the risk and what their impact will be.

Dr Obadia: May I add a comment. From the beginning of this session we have heard, with the first presentation by our colleague, that the larger the group of patients involved in the evaluation, the better is the prediction. On the opposite side, the smaller the group, the worse the prediction. But as a surgeon, when I am facing a patient, I have the smaller group, we can imagine, which is a group of one patient, and as a consequence, do I have to conclude that there is absolutely no interest in using any score as a surgeon when facing one patient?

Dr P. Sergeant(Leuven, Belgium): It is a very fundamental question which you have just raised. Indeed, what we are doing in reality is we are deciding about a patient's life and a patient's therapy based on a positive predictive value of 10, 12, 16%. So your comments are very correct, and they fit in with comments that I made earlier.

Your question about prediction, indeed you are faced with one patient. My mentors, John Kirklin and Gene Blackstone, told me that you will never be able to predict an individual patient; an individual patient is always at random. But what you will be able to get information about is a cohort of patients: 100 patients, 1,000 patients, that have exactly the same number of variables and the same values for these variables. So that information you can get. And you can get that with a rather small uncertainty.

And then I come back to a criticism that I made earlier, which is that if that cohort is described in nominal variables, yes/no variables, it is uncertain. And the more you are able to describe it in continuous variables (continuous ejection fraction, continuous pulmonary function, and so on), the more refined your information will be on the cohort. But it will never be about the individual patient.

Dr S. Nashef(Cambridge, UK): As Dr. Obadia so elegantly said, when you are faced with a very small group of one patient, all risk models are 100% wrong, because that patient will either live or die, and any model that predicts anything between 0 and 100% is completely wrong. However, knowing that a cohort of 1,000 similar patients had a 5% mortality may help the patient in being guided to a decision.

I just want to separate something which I think may lead to some confusion. All of these risk models for open heart surgery are risk models for open heart surgery. They are useful in perhaps identifying high-risk patients so that TAVI can be considered, but they were never designed to distinguish between mortalities for TAVI. As we stated this morning, what makes TAVI risky is not what makes open heart surgery risky. So, Dr. Haensig, I fully agree with you: a TAVI risk model is well overdue.

Author notes