Surgical management of congenital heart disease: evaluation according to the Aristotle score

Abstract

Objective: The Aristotle basic complexity (ABC) score (1.5–15 points) is the sum of potentials for early mortality, morbidity and anticipated surgical technique difficulty. The Aristotle comprehensive complexity (ACC) score (1.5–25 points) is the sum of ABC score and patient-adjusted complexity score; it comprises six complexity levels. We used the ACC score to evaluate quality in surgical management of congenital heart disease. Methods: Procedures performed in year 2002 and 2007 were analysed. Proportion of procedures requiring at least 1 week of stay in the intensive care unit was chosen as the marker of morbidity. We adopted threshold duration of 120 min for cardio-pulmonary bypass (CPB) cases and the same duration for operations without CPB as surrogate of surgical technical difficulty. The ACC scores were correlated to mortality, morbidity and technical difficulty. Results: This study included 758 patients who underwent 787 primary procedures. The mean ABC and ACC scores amounted to 7.61 ± 2.46 and 9.51 ± 3.84. Early mortality was 3.05% (24/787), 95% confidence interval (CI): 1.97–4.51%. Zero at ACC levels 1 and 2, it increased from 1.2% (2/161) for level 3 up to 22.2% (2/9) for level 6. Morbidity index was evaluated at 25.9% (204/787), 95% CI: 22.9–29.1%. 1.9% at level 1, it escalated up to 77.8% at level 6. Index of technique difficulty was estimated at 35.2% (277/787), 95% CI: 31.8–38.6%, ranging from 4.8% for level 1 to 66.7% for level 6. A high correlation was found between the ACC scores and mortality, indices of morbidity and technique difficulty, Spearman’s correlation coefficient r being 0.9856, 1 and 0.9429, respectively. Mortality (p = 0.037) and morbidity (p = 0.041) were lower in year 2007 than in 2002 with ABC (p = 0.18) and ACC (p = 0.37) surgical performance being not significantly different. Conclusions: The Aristotle score is still under development. Morbidity evaluation should be ideally based on observed postoperative complications; estimation of surgical technical difficulty chosen in this study may not be generalised. Nevertheless, the actual Aristotle comprehensive complexity score, as evaluated in its three components, accurately determined the outcome of surgical management of congenital heart disease. It appears to be an adequate tool to evaluate quality in paediatric cardiac surgery, over time.

1 Introduction

Evaluation of the quality of care has become a duty of modern medical practice. This is particularly important for surgical disciplines, especially those prone to substantial operative morbidity and mortality, such as heart surgery. There is a demand from different partners (patients and their families, referring physicians as well as insurance companies, etc.) to clearly report outcomes. This would lead not only to better disease management, but also to increasing accountability. Institutions (surgical units) could be compared, thus facilitating informed choice from health-care seekers.

Although results of repair or palliation of congenital heart disease have steadily improved in the last 2 decades, paediatric cardiac surgery remains a subspecialty with number of complex high-risk lesions. The surgeon is confronted with a wide range of operations, with many of them done rarely. Simple analysis of mortality rates may be misleading when other factors are not considered, complexity of cases in particular. A cardiac unit performing many high-risk procedures might poorly reflect on overall mortality rates when compared to a centre dealing essentially with low-risk malformations. Therefore, it is essential to use an adequate method to assess surgical performance based on reliable and comparable data.

The Aristotle score [1] was developed from 1999 to September 2003 by a panel of international expert paediatric cardiac surgeons representing 50 centres and 23 countries, as a tool to evaluate performance in surgical management of congenital heart disease. This consensus-based risk stratification model comprises two scores:

1. The Aristotle basic complexity score (ABC score), which is a procedure-adjusted complexity (1.5–15 points), is the sum of potentials for early mortality, morbidity (intensive care unit (ICU) length of stay), and anticipated surgical technique difficulty (each, 0.5–5 points).

2. The Aristotle comprehensive complexity score (ACC score) (1.5–25 points) is the sum of the basic score and patient-adjusted complexity score (0–10 points). This includes procedure-dependent factors (0–5 points) and procedure-independent factors (general, clinical, extracardiac and surgical: 0–5 points). The Aristotle score recognises six levels of complexity: from the less complex level 1 with 1.5–5.9 points to the most complex level 6 with 20.1–25 points. As stated by Lacour-Gayet et al. [1], the complexity score is a constant, at a given time for a given procedure in a given patient, whatever the centre and its global location. Performance is defined as complexity score (constant) multiplied by outcome (variable). Surgical (operative) performance therefore can be calculated as complexity multiplied by hospital survival. It may be estimated for each surgeon, surgical unit and institutions, thus allowing comparison. The Aristotle score is still under validation. Based on objective data of postoperative morbidity and mortality from large databases, an Aristotle average complexity score is actually under development [2].

The aim of this study was to analyse outcome of procedures performed in our unit, to determine groups of higher-risk patients and to estimate surgical performance over time with the help of the actual Aristotle scoring, in order to adapt our surgical strategy and further improve care quality.

2 Patients, procedures and methods

2.1 Procedures

All surgical procedures performed in our unit at 5-year interval (years 2002 and 2007) were analysed. Our centre carries out the whole spectrum of congenital heart surgery, except for cardiac transplantation.

The International Congenital Heart Surgery Nomenclature was used [3]. Excluded from this study were surgical closure of patent ductus arteriosus (PDA) in premature newborns, primary extracorporeal membrane oxygenation (ECMO) and cardiac pacemaker implantation, as in previous studies [2,4]. For each hospital stay of the patient, only one main (primary) procedure was determined according to Aristotle scoring: that is, operations involving two or more procedures during the same operating time, as well as re-operations carried out in the same hospital stay, were assigned to the procedure having the highest ABC score [1,4], the other being considered as secondary procedures or associated procedures.

2.2 Data collection

Year 2002 data were collected retrospectively. They included, particularly, demographic characteristics (age and body weight at time of surgery), cardiac and non-cardiac diagnoses, type of procedure(s), duration of operation, data of cardiopulmonary bypass (CPB, duration of mechanical ventilation, ICU and hospital stay and any postoperative complication, reintervention and death. Procedure-dependent and procedure-independent factors (Appendix B3 and B4 of reference [1]) were recorded to allow ACC score estimation. Year 2007 data were steadily available since May 2005, ABS and ACC scores are systematically determined for all patients undergoing heart surgery in our unit as part of the validation project of the Aristotle score, using a database supplied by the Aristotle Institute, Denver, Colorado, USA (http://www.aristotleinstitute.org). This database is based on the Aristotle score final November 30 2004 version, provided by Aristotle Institute as well and contains the aforementioned parameters.

2.3 Estimation and evaluation of Aristotle scores

Each main procedure was assigned a basic and a comprehensive score and grouped in one of the six complexity levels of the Aristotle score. Thereafter, each of the three components of the Aristotle score was individually evaluated as follows:

1. Mortality: Early mortality is defined as death occurring in the first 30 postoperative days or during hospital stay. It was correlated individually to each comprehensive score and globally to complexity levels.

2. Morbidity: The incidence of postoperative complications was first recorded for each complexity level and correlated to duration of mechanical ventilation, ICU and hospital stay. ICU stay equal or longer than 7 days was retained as a marker of morbidity. An index of morbidity was calculated by dividing the number of procedures with ICU stay of at least 1 week duration (n) by the total number of procedures (N): n/N. This proportion, expressed in percentage, was then used to further correlate morbidity to ACC scores and complexity levels.

3. Surgical technique difficulty: To be able to retrospectively assess this parameter, we empirically chose duration of CPB and operation duration for procedures carried out without CPB. For procedures that needed CPB, CPB duration of more than 120 min was selected as surrogate for technique difficulty. For those without CPB, operation duration longer than 120 min was also accepted. An index of technique difficulty was calculated by dividing the number of CPB procedures with CPB duration >120 min (n1) plus the number of non-CPB procedures with operation duration >120 min (n2), by the total number of procedures (N): (n + n2)/N. This index (%) was again correlated to ACC scores and complexity levels.

A 95% CI was calculated for early mortality, index of morbidity and index of surgical technique difficulty. Procedures, groups of patients and lesions having figures above 95% CI were considered to be potentially higher risk, and those under 5% as lower risk.

Year 2002 and year 2007 data were compared, with emphasis on surgical performance.

2.4 Data analysis

Procedures that were carried out at least 8 times, cases of year 2002 and year 2007 combined, were individually analysed, the others globally. The software GraphPad Prism (San Diego, CA, USA) was used to determine nonparametric correlation (Spearman’s correlation coefficient r) between Aristotle scores and mortality, morbidity and technique difficulty, and to depict regression (goodness of fit r²) line for the three parameters, with their respective 95% CI. CIs for proportions were computed by means of StatMate (GraphPad). Means are given with standard deviation (SD) and percentages with 95% CI when indicated. Comparison of means between two groups was performed with the independent samples t-test, between more than two groups with one-way ANOVA and for dichotomous variables with the chi-square test. The significance level was set at a p-value of ≤0.05.

3 Results

3.1 Characteristics of patients and procedures

In the 2 years under review, 758 patients underwent 787 primary procedures. Table 1 summarises the main characteristics of patients and outcome according to age groups. Most procedures were carried out under the age of 1 year, 146 (18.6%) operations taking place in the neonatal period. Weight at the time of repair was less than 2.5 kg in 22 patients. The great majority (n = 722: 91.7%) required CPB, which lasted an average of 109 ± 71 min. Mean durations of postoperative mechanical ventilation (55 ± 121 h), ICU stay (6 ± 16 days) and hospital stay (19 ± 11 days) were highly significantly different (p ≪ 0001) between the four age groups, the numbers being much higher in the neonatal period. There were 24 early (hospital) deaths, a mean mortality of 3.05% (1.97–4.51%), and 501 complications were encountered. Higher incidence was observed in newborns with a mortality of 9.6% (14/146) and more than one complication per case (151/146).

3.2 Aristotle score figures

Ventricular septal defect (VSD) patch repair (ABC score = 6) was the most frequent procedure: 8.9% (70/787) of cases, followed by aortic valvuloplasty (ABC score = 8): 6.0% (47/787). Table 2 lists procedures that were carried out at least 8 times. Mean ABC score was 7.61 ± 2.46, the score ranging from 3 (atrial septal defect (ASD) primary closure: n = 32) to 15 (hypoplastic left heart sysndrome (HLHS) biventricular repair: n = 3).

Patient-adjusted independent factors used to estimate ACC scores are recorded in Table 3 . Factor with the highest points (4 points) was extreme prematurity ≪32 weeks: n = 9, and the most frequently met was redosternotomy (2 points, n = 121). Overall mean ACC score was estimated at 9.51 ± 3.84, range 3–24.5. Most procedures (39.1%: 308/787) were assigned to level 4 ACC score: 10–15 points (see Table 4 ). The highest complexity level 6 (20.1–25 points) was reached on nine occasions: 1.1% of cases.

3.3 Aristotle score and mortality

With a mortality of 3.05%, ABC and ACC surgical performance attained 7.38 ± 2.38 and 9.22 ± 3.72, respectively. Mean ACC score for the 24 patients who died was 14.62 ± 3.74, median 13.75, range 8–22.50. Table 2 indicates operative mortality per procedure. As shown in Table 4, there was no mortality for procedures with ACC scores at levels 1 and 2. Mortality was highly correlated with ACC scores: increasing with raising scores, from 1.2% (2/161) for level 3 cases to reach 22.2% (2/9) for those in level 6. Spearman’s correlation coefficient r was 0.9856 (p = 0.0028). The best computed regression line was a sigmoid as illustrated by Fig. 1A , r² being estimated at 0.9856.

3.4 Aristotle score and morbidity

Positive correlation was found between the incidence of postoperative complications and the durations of mechanical ventilation, ICU and hospital stay, r² being calculated at 0.8858, 0.9057 and 0.9746, respectively. Table 5 lists the complications that occurred, the most frequent being arrhythmia (n = 60). Individual indices of morbidity per procedure are given in Table 2. Global index of morbidity was estimated at 25.9% (204/787) (22.9–29.1%): that is, 25.9% of cases stayed, postoperatively, at least 7 days in ICU. As illustrated by Table 6 and Fig. 1B, there was an almost perfect correlation between the estimated index of morbidity and ACC scores, r Spearman’s coefficient being equal to 1, and r² for the linear regression line approaching 1: 0.9956.

3.5 Aristotle score and surgical technique difficulty

Calculated indices of technique difficulty (≥120 min. CPB duration or non-CPB operation duration) for each procedure are displayed in Table 2. The index for the whole cohort amounted to 35.2%: 277/787 (31.8–38.6%). It also correlated well with ACC scores (r = 0.9429). The best computed regression line was a sigmoid as illustrated by Fig. 1C, r² being calculated at 0.9756.

3.6 Aristotle score and higher-risk groups

The following procedures were encumbered with mortality, index of morbidity and of surgical technique difficulty (all three parameters) that were superior to the calculated 95% CI: in the order of decreasing number of cases, Norwood procedure, arterial switch operation, TCPC lateral tunnel fenestrated, arterial switch operation and VSD repair, TAPVC repair and truncus arteriosus repair. The other interventions with one or two values above 95% CI can be viewed in Table 2. Neonates constituted a higher-risk group with all three parameters being situated above 95% CI (see Table 6). Based on Aristotle scoring, ACC levels 5 and 6 clearly fulfilled higher hazard criteria, whatever the operation, at any age, as evidenced by Table 6.

3.7 Aristotle score and year 2002/year 2007 comparison

Aristotle scoring does not show any significant difference between ABC and ACC scores and corresponding surgical performances. But Table 7 testifies to clear improvement after 5 years in observed mortality and in estimated morbidity, coupled with shortening of operation times.

4 Discussion

The Aristotle score is an expert consensus-based risk stratification model that is still under validation. Assignment of a basic score to any procedure is straightforward. Figures of ABC scores and performance reported in this study are therefore accurate. On the contrary, comprehensive Aristotle scoring requires a thorough determination of procedure-dependent and procedure-independent factors. Some of these can be missed, particularly in retrospective search. It has then to be assumed that ACC scores and performance in this work are most probably underestimated.

Some authors have analysed accuracy of ABC score for classifying the mortality [2,4–6] and morbidity [2,4] potential of congenital heart surgery operations. This publication is the first to evaluate ACC score for the whole range of surgical procedures in a paediatric cardiac unit and to consider all three aspects of the Aristotle score, including anticipated surgical technique difficulty.

Mortality numbers cannot be challenged. However, morbidity estimation bears uncertainty. It should be ideally based on observed postoperative complications. These should be weighed according to their severity: for example, their potential to affect patient’s quality of life. No scale of complications is actually available to measure how (much) they influence postoperative outcome. O’Brien et al. [4] chose prolonged hospital stay (>21 days) as a marker of morbidity and observed a significant positive correlation with the ABC score. Clarke et al. [2] recently discussed the whole matter and proposed that morbidity score be constituted of the following four components: postoperative hospital length of stay, postoperative time on the ventilator, postoperative ECMO and/or ventricular assist device time, and major complications (re-operation, permanent pacemaker for atrio-ventricular block, nerve palsy, neurological disorders and dialysis). The algorithm is under development and the first significant values for the complete index are not expected before 2011. Meanwhile, they presented a morbidity indicator based on the sum of length of stay in the hospital and time on ventilator. We found a high correlation between the number of postoperative complications and each of the three durations of mechanical ventilation, ICU and hospital stay. In agreement with the original Aristotle score principles [1], we selected ICU stay as morbidity surrogate, length of 1 week or more being defined as increased morbidity, even if correlation with hospital length of stay was found to be more positive. The finding in this study of a Spearman correlation coefficient r of 1, and r² of almost 1 for corresponding linear regression indicates an almost perfect correlation between this index of morbidity and ACC scores. Therefore, we are of the opinion that this index may be further used until the projected morbidity index becomes available and accepted. It is clear that this index is distorted in those centres where intermediate care units function.

The evaluation of surgical difficulty, fundamentally a subjective notion, is even more questionable. How can it be estimated retrospectively? Variability can also be expected according to surgical experience, social environment, surgical team, etc. We hypothesised that procedure duration somewhat reflected technical difficulty (problems) encountered while performing surgery. After analysing operations and CPB times, we fixed the threshold at 120 min for CPB time and adopted the same limit for operations without CPB. We agree with Clarke et al. [2] that this threshold may not be generalised and applied everywhere. It should reflect each surgeon/unit’s operating speed and be adapted to local circumstances. The index reported here should be therefore accepted with caution as other duration thresholds could be elaborated in different settings. Nevertheless, despite evident drawbacks, the method used in this study permitted us to quantify (retrospectively) surgical technique difficulty. It is to be noted that, prospectively while evaluating the Aristotle score, the surgeon is asked to determine, subjectively, this difficulty, after each procedure, awarding 1 (elementary difficulty) to 5 (major difficulty) points.

First publications on ACC score showed significant correlation with early mortality after repair of truncus arteriosus [7], Norwood procedure [8,9] and CPB procedures in low weight (≪2.5 kg) patients [10]. Mortality was particularly high when ACC score reached 19.5 [10] or 20 [7,8] points. This study confirms these findings for all procedures with r for mortality correlation of almost 1 (0.9856). It gives corresponding mortality figures for all ACC levels. It further estimates corresponding morbidity and surgical difficulty, again with escalating numbers pertaining to higher complexity levels and with also excellent correlation coefficients (r = 1 and 0.9429). ACC score appears to be an adequate tool to analyse outcome and to compare results, thus to monitor quality of surgical management of congenital heart disease.

Although global figures given by this work are accurate, individual values, in particular for procedures with small numbers, have to be considered as indicative. Large series such as the one of Clarke at al. [2] would give more reliable results.

ACC scoring clearly identified procedures with ACC levels 5 and 6 as higher-risk procedures, and neonates as the most endangered group of patients. Efforts should be devoted to improve outcome for these cases. This includes preoperative estimation of ACC score to alert the surgical team and to counsel parents accordingly. In our centre, the introduction of ECMO at the end of year 2002 might partially explain the significant improvement in mortality observed in year 2007. Quality improvement is also illustrated by the lower index of morbidity encountered. However, a specific score to quantify and evaluate postoperative complications is needed in order to make a more adequate comparison over time.

5 Conclusion

Comprehensive Aristotle score, in its three components (mortality, morbidity and surgical technique difficulty), accurately determined outcome of surgical management of congenital heart disease. It appears to be actually an adequate tool to analyse and compare results, and therefore to monitor quality care in paediatric cardiac surgery, over time.

References