Performance of three preoperative risk indices; CABDEAL, EuroSCORE and Cleveland models in a prospective coronary bypass database Free

Abstract

1 Introduction

The median age of the population is increasing, creating a larger pool of patients suffering from coronary artery disease [1,2]. This may potentially require invasive procedures, including coronary artery bypass (CAB) surgery. The ageing population and changing patterns in coronary artery disease have led to a situation where the age of CAB patients is also increasing, and higher age is also associated with increased comorbidity. Thus, there will be an increased risk of postoperative complications. The prediction of postoperative morbidity and mortality is useful for many reasons, for example in measuring the quality of care and estimating the need for resources. A number of risk indices [3–10] have been developed for the prediction of postoperative mortality, including the EuroSCORE [3] and the Cleveland model [4]. Few indices have been developed solely for the prediction of morbidity; one such is the CABDEAL model [5].

Since the relative utility of the risk indices for different purposes may not be the same, the purpose of the present study was to compare three different types of risk model, the CABDEAL, Cleveland model and the EuroSCORE, in the prediction of morbidity on the one hand and mortality on the other in a prospective database.

2 Materials and methods

The data were obtained prospectively from Tampere University Hospital between May 1999 and November 2000. The entire patient sample consisted of 1132 consecutive coronary artery bypass graft (CABG) patients. The study was approved by the institutional review board of Tampere University Hospital and every patient gave written informed consent for this study. The data were not traceable to the practitioners involved in their care. The database included patient age and gender, admission status, (i.e. elective, urgent or emergency), preoperative diagnoses and laboratory values, primary and secondary procedures, postoperative complications, in-hospital mortality and discharge status (i.e. to the secondary referral hospital or rarely to home). The postoperative length of stay (POS) in primary hospital was also included in the data.

There were 827 (73%) men and 305 (27%) women in the study. The mean age for the entire cohort was 64.6 years (SD 9.65). There were 1081 first-time CABG procedures and 49 reoperations. The CABG operations were carried out on-pump in 1017 cases, 95 patients had off-pump surgery and 20 underwent a minimally invasive MIDCAB procedure. Because the majority of this patient population had regular on-pump surgery and the groups for the OPCAB and MIDCAB were so small it was decided to treat the whole patient sample as one group. All patients were prospectively scored according to the initial CABDEAL, EuroSCORE and Cleveland models. Tables 1–3 summarize the risk factors used in these models.

Table 1

The CABDEAL scoring system

Open in new tabDownload slide

Table 2

The Cleveland risk-scoring system

Open in new tabDownload slide

Table 3

The EuroSCORE scoring system

Open in new tabDownload slide

2.1 Data analysis

The CABDEAL model [5] was developed using a Bayesian approach [11], and it includes seven preoperative risk factors which are assigned point values: renal dysfunction, advanced age, obesity, diabetes mellitus, emergency surgery, arrhythmia and/or unstable angina or previous AMI, and chronic obstructive lung disease. The exact criteria and point value assignments and the description of the model are detailed in Table 1 and in  Appendix A. The maximum possible risk score is 10. A score of 3 or more is associated with increased morbidity, the outcome criteria for which were in the original model the following: at least one severe complication postoperatively and POS of ≫11 days, or at least two mild complications postoperatively; or death during the hospital stay. Due to changes in treatment policies the POS limit for increased morbidity was now lowered to 8 days. The patients in this study were transferred to a local community hospital usually on the sixth or seventh postoperative day if they developed no major complications. Severe postoperative complications included reoperation due to bleeding or low cardiac output, mediastinitis, pneumonia, or prolonged ventilatory support (≫36 h) or intensive care unit (ICU) stay longer than 3 days, stroke (cerebrovascular accident), acute renal failure requiring dialysis or a more than 50% increase in serum creatinine level (compared to preoperative values), severe cardiac failure requiring inotropic support or intra-aortic balloon pump insertion or perioperative myocardial infarction (MI), severe arrhythmia (ventricular fibrillation or asystole) or death. Mild complications included superficial wound infection (leg or sternum), atrial fibrillation or other mild supraventricular arrhythmia.

The preoperative factors and score values for the EuroSCORE and Cleveland models are listed in Tables 2 and 3. These models are developed using logistic regression analysis. The EuroSCORE model was originally validated only for the prediction of mortality. The Cleveland model is designed to predict both morbidity and mortality. The score level for significantly increased mortality (≫10%) in EuroSCORE is 6 [3], and in the Cleveland model also 6 [4]. For increased morbidity the score level is 4 in the Cleveland model [4]; the EuroSCORE was not originally validated for morbidity and has no score limit or outcome criteria definitions for increased morbidity. Here the same outcome criteria were applied for the EuroSCORE and the Cleveland model for morbidity. Outcome criteria for increased morbidity in the Cleveland model comprised one of the following: central nervous system complication (stroke), cardiac complication (perioperative MI, need for mechanical assistance device, intra-aortic balloon pump), acute renal failure requiring dialysis, serious infection (mediastinitis, sepsis), return to surgery due to bleeding, prolonged mechanical ventilation (≫3 days) or death.

A likelihood ratio method and Bayesian approach [11] were used to assess the relationship between preoperative risk factors and postoperative outcomes in the CABDEAL model. The area under the receiver operating characteristic (ROC) curve (C statistic) [12] was reported as an indicator of the predictive value of the CABDEAL, EuroSCORE and Cleveland indices for various outcomes.

Univariate analysis and χ²-tests were used to establish the significance of the preoperative risk factors in each model. The preoperative risk scores were calculated for the CABDEAL, Cleveland and EuroSCORE models and the sum of the weighted scores were listed for every patient.

3 Results

The demographic data and baseline patient characteristics for the study cohort are presented in Table 4 . Arterial hypertension was listed in 616 of the patients (54.4%). Unstable angina was diagnosed in 317 patients (28%) and previous MI (≪90 days) in 249 (22%). Diabetes mellitus was present in 225 patients (20%), arrhythmia was reported preoperatively in 106 (9.4%) and chronic pulmonary disease in 82 (7.2%). There were 42 emergency operations (3.7%), 401 urgent (35.4%) and 689 elective cases (60.9).

Table 4

Preoperative patient characteristics

Open in new tabDownload slide

Major postoperative complications developed in 268 (23.6%) patients. Pneumonia occurred in 12 (1.0%), haemorrhage requiring evacuation in 44 (3.9%), and severe cardiac failure in 16 (1.4%). Postoperative stroke was diagnosed in 29 patients (2.6%). Perioperative MI was recorded in 80 (7.1%) patients, and three patients needed intra-aortic balloon pump support. Acute renal failure with 50% increase in serum creatinine levels was diagnosed in 102 patients (9.0%), and 24 of them needed hemodialysis. Mediastinitis was diagnosed in six patients (0.5%) and sternal or limb wound infections in 21 patients (1.8%). There were 38 deaths during the hospital stay (3.4%). Mild postoperative complications (including atrial fibrillation) occurred in 383 patients (33.8%). Tachycardia and brady-arrhythmia were recorded in 123 cases (10.9%) postoperatively. Ventilation time over 36 h was recorded in 49 (4.3%) patients and 191 patients (16.9%) stayed in the ICU for 3 or more days. The mean POS was 6.9 days (SD 3.8). The majority of the patients (75%) had a favourable outcome without major complications.

The area under the ROC curve for morbidity was 0.772 for the CABDEAL, 0.694 for the EuroSCORE and 0.686 for the Cleveland model. The ROC curves for mortality were 0.711, 0.826 and 0.858, respectively. Table 5 shows the ROC area under the curve values in the three models.

Table 5

C statistics (ROC areas under the curve) of the EuroSCORE, Cleveland and CABDEAL models for mortality and morbidity

Open in new tabDownload slide

4 Discussion

In this study we investigated the predictive value of three preoperative risk models in the prediction of morbidity and mortality in a prospective database of 1132 CAB patients. Previously developed risk indices, the CABDEAL, the EuroSCORE and the Cleveland models, were used to quantitate the preoperative risk for morbidity and mortality. Known preoperative risk factors for postoperative complications, e.g. diabetes, chronic lung disease and renal dysfunction, were recorded in the database. This database covered all the single risk factor variables and their values. Also postoperative morbidity, mortality and length of stay data were consistently recorded. There were no missing values in this database.

The main preoperative risk factors observed here remain consistent with the findings in most previously published risk stratification studies for this patient population [6–10,13]. No consensus exists as yet regarding the universal weighting of the various factors and the calculation of the risk. Most of these indices were derived using multivariate logistic regression [3,4,7–9,13] and are not convenient bedside models for routine clinical practice. They tend to require complex mathematical calculations to determine individual patient risk and many of them have not been validated for single patient use, only for use in large patient populations. The CABDEAL index, developed using a Bayesian approach [11], seems better suited for individual patient calculations by merit of its simplicity and moderate predictive value for morbidity as demonstrated in this study. As a consequence of changing clinical practice we updated the outcome criteria for the POS in the CABDEAL model. In our original model the POS limit for increased morbidity was 11 days, patients staying in the hospital in fact for an average of 12 days. Now that hospitals tend to ‘fast-track’ their patients the POS is decreasing dramatically. Our criteria for morbid outcome in the CABDEAL model in this study were 8 or more days postoperative stay in the hospital and at least one concomitant major complication. Otherwise there were few differences between the present material and the original patient cohorts in terms of CABDEAL, Cleveland and EuroSCORE. In this present study there were 115 patients who did not have cardiopulmonary bypass for their CAB operation. The risk profiles of these patients did not seem to differ from the basic database population. Because the off-pump and MIDCAB groups were so small we preferred not to separate these patients from the study group.

Definitions for morbid outcome criteria seem to vary from study to study. Some models, for example the Cleveland model, have already defined their original morbid outcome criteria [4]. Because the EuroSCORE was not originally validated for morbidity it has no exact outcome criteria for morbidity. That may explain why EuroSCORE does not perform very well in the prediction of morbidity [14]. The outcome criteria for morbidity should be determined in the original database from which the scoring system was derived, otherwise the score should not be used to predict morbidity. Different authors tend to use their own outcome criteria in comparing the performances of the different risk models in the prediction of morbidity [14–16]. The CABDEAL model was originally derived for the prediction of morbidity in the first-time CAB population. The rate of mortality in the original cohort from which the scoring system was derived was very low: six out of 386 patients died (1.6%) [5]. For this reason the CABDEAL model was originally validated only to predict morbidity, not mortality. It has been shown elsewhere that the risk factors for morbidity are not the same as those for mortality [14]. This may partly explain the differences in the performance of the different risk models.

Finally, the most important issue in the every day clinical practise is not which model to use. The chief concern is to be able to stratify patients according to their risk profiles into high-risk and low-risk populations and to make decisions based on this information for resource allocation and probable preoperative treatment designed to reduce if possible the risk of morbidity and mortality.

Based on this study both the EuroSCORE and the Cleveland model are preferable for the prediction of mortality in patient groups. The CABDEAL can be recommended for bedside preoperative prediction of morbidity in CAB patients.

If hospitals were able to accurately stratify patients into low-, medium- and high-risk groups for morbidity and mortality, this information could be of great value. Hospital resources could be maximized by optimal scheduling of complex cases to avoid overcrowding of ICUs at any one time. Also the individual patient could benefit from risk stratification in being brought preoperatively to an optimal condition (concerning diabetes, chronic lung disease, renal function, angina, arrhythmia etc.) after evaluation of risks has been made. Risk stratification should therefore be undertaken as early as possible after the decision to operate has been made. Then surgical, anaesthesia, and ICU teams, family members and the patient would benefit most from risk assessment.

The Bayesian approach states how posterior probability P(D|x) is calculated using the prior probabilities P(D) and P(D′), which states how common the diagnosis D and its absence are in the population. The probabilities of symptoms given the diagnosis P(x|D) and P(x|D′) are commonly estimated by their frequencies in learning material. In the case of a two-valued outcome, the Bayesian approach is commonly developed to give a likelihood ratio. The likelihood ratio (L) of a set of observed symptoms of properties x_i is written as the product of the probability ratios of each x_i, also known as risk ratios in epidemiology:

Postererior probability=L(x)/[1+L(x)]

If the prior probabilities P(D) and P(D′) are far from 0.5, their effect is taken into account by multiplying L by an empirical coefficient r based on earlier knowledge or on learning material [11]. The likelihood ratios are expressed in the tables as ten times ten based logarithms of the original values. A constant 10 has been added (10×(1+log (L))). If any risk factor value is missing an estimate of 10 is used, which will equal a likelihood value of 1.0.

A.1 Step-by-step description of the three models developed

We have developed several models for different purposes: a longer model with 15 variables for research purposes (for testing outcomes in patient populations) preferably with the use of a computer, and a second model with seven variables meant to be used with a handhold calculator in predicting the outcome for one patient. We have further developed a version that uses a summation model rather than a multiplicative version. In that manual model each variable has only two values, ‘normal’ and ‘at risk’. This model is meant to be used bedside for every patient scheduled for CABG surgery without any calculator. This model is a analogy for the APGAR score. We have used in this study the short seven-variable model.

References