Comprehensive assessment of frailty score supplements the existing cardiac surgical risk scores

Abstract

 

Open in new tabDownload slide

OBJECTIVES

The commonly used cardiac surgery risk scores, European System for Cardiac Operative Risk Evaluation II and Society of Thoracic Surgeons score, are inaccurate in predicting mortality in the ageing patient population and do not include the biological age. This requests a need for a new risk score incorporating frailty. The aim of this study was to compare the prediction of mortality and the additive effect of comprehensive assessment of frailty score and the shortened version, frailty predicts death one year after elective cardiac surgery test on the existing risk scores.

METHODS

Six hundred four patients undergoing cardiac surgery and aged ≥65 years were included in this prospective observational study. These frailty scores are based on minor physical tests. We compared these frailty score predictions of mortality and their added value to the existing risk scores evaluated by concordance-statistics (C-statistics), integrated discrimination improvement and net reclassification improvement.

RESULTS

The median age was 73 years (21% female). C-statistics showed that comprehensive assessment of frailty score with a value of 0.69, frailty predicts death one year after elective cardiac surgery test 0.68, Society of Thoracic Surgeons score 0.70 and European System for Cardiac Operative Risk Evaluation 0.64. Frailty assessment, added to the existing risk scores, significantly improved integrated discrimination improvement up to 0.05, and net reclassification improvement up to 0.04. Frailty assessment also increased the C-statistics, but this did not reach statistical significance.

CONCLUSIONS

Frailty scores are as good as the existing risk scores for the prediction of mortality in patients undergoing cardiac surgery. Added to the existing scores, frailty assessment improves the C-statistics and integrated discrimination improvement over time.

Clinical trials registration number

NCT02992587

INTRODUCTION

Risk scores for the prediction of mortality and complications in patients undergoing cardiac surgery have been widely used for >30 years [1]. A solid risk prediction model thus constitutes a strong tool in quality assessment and improvement [2]. The 2 most frequently used risk scores of mortality in patients undergoing cardiac surgery are the European System for Cardiac Operative Risk Evaluation (EuroSCORE) II [3] and The Society of Thoracic Surgeons (STS) score [4]. These scores are validated to estimate the short-term mortality [3, 4]. Both scores incorporate variables that are related to patients’ demographics, comorbidities, cardiac status and operative procedures and do not incorporate the biological age. Not incorporating the biological age might explain why the scores become inaccurate in risk estimation [3, 5, 6]. The biological age, also referred to as frailty, has had increased attention during the last decade, specifically its potential predictive ability, but the focus has also been on how to best assess frailty [7]. Several studies have investigated the assessment of frailty in patients undergoing cardiac surgery, and a variety of measures have been used [8]. Previous studies have provided evidence that frailty is a predictor of mortality and might have the potential to improve the risk prediction of the existing risk scores [2, 9–14]. We have previously reported the comprehensive assessment of frailty (CAF) score’s ability to predict the 30-day and 1-year mortality in patients undergoing cardiac surgery [15, 16]. The study found that the CAF score is a good predictor of mortality after undergoing cardiac surgery. The same applies to the evaluated shortened version of the CAF score: “Frailty predicts death one year after elective cardiac surgery test” (FORECAST).

The purpose of the present study was to compare the predictive ability of mortality of the CAF score and FORECAST with the existing risk scores EuroSCORE II and STS. Second, we wanted to investigate the effect of the CAF score and FORECAST on risk prediction when added to the existing risk scores, EuroSCORE II and STS score, in patients undergoing cardiac surgery. The hypothesis of such a comparison was that it should illustrate methods that could improve and optimize risk scores for patients undergoing cardiac surgery.

METHODS

The study population has previously been described [15, 16]. In brief, between February 2016 and January 2018, patients aged ≥65 years undergoing elective cardiac surgery (coronary artery bypass graft and/or valve replacement/repair) at our institution (Department of Cardiothoracic Surgery, Rigshospitalet, Copenhagen University Hospital, Denmark) were included in this prospective observational study. Our exclusion criteria were: (i) emergency surgery (<24 h), (ii) clinically unstable patients, (iii) severe neuropsychiatric impairment, (iv) no cooperation (psychiatric diagnosis) and (v) re-operations. Baseline demographics and clinical information were collected prospectively in all patients. The 2 most frequently used cardiac surgery risk scores, EuroSCORE II and STS score, were calculated preoperatively through their website [2, 9] on all patients. The frailty scores, CAF score and FORECAST were assessed on all patients before cardiac surgery. Searching the Danish Central Civil Register in January 2020 constituted the last follow-up for survival or death. No patients were lost to follow-up.

CAF score

The CAF score consists of 4 parts with different set items. Part 1 is based on the Fried criteria measuring exhaustion, daily physical activity, grip strength and gait speed. Part 2 is a physical test of balance and body control. Part 3 consists of laboratory tests, including serum creatinine, serum albumin and forced expiratory volume in the first second. The fourth and last part was the surgeon’s evaluation of the patients’ frailty by the clinical frailty scale (CFS) score (‘eyeball test’). The CAF score is described in more detail in earlier studies [15, 17]. The score ranges from 0 to 35, and the patients are deemed frail if the score is 11 or above.

Forecast

The shortened version of the CAF score, FORECAST, is based on 5 items from the CAF score: chair rise, stair climb, CFS, exhaustion and serum creatinine level [9]. The score ranges from 0 to 14, and the patients are deemed frail if it is 5 or above.

Outcomes

In this study, the primary outcome was to evaluate the ability of the CAF score and FORECAST to predict mortality compared to the existing risk scores, EuroSCORE II and STS score, in patients undergoing cardiac surgery. The secondary outcome was to evaluate the additive effect of the CAF score and FORECAST to the existing risk scores EuroSCORE II and STS scores.

Statistical analysis

Demographics and surgical data are presented as median with the first and third quartile or range or means with standard deviations in the case of continuous variables and as numbers and percentages in the case of categorical variables. The median follow-up was estimated by the reverse Kaplan–Meier method.

Cox proportional hazard regression was used to analyse the association between the risk scores and all-cause mortality after cardiac surgery. This is presented as a hazards ratio, followed by a 95% confidence interval (CI). The proportional hazards assumption and functional forms of predictors were assessed graphically and numerically using cumulative Martingale residuals. All risk and frailty scores were log-transformed with base 2 to meet the assumption of linearity.

The association between each item in FORECAST and all-cause mortality adjusted for STS and EuroSCORE II, respectively, was analysed using multivariable cox regression. In addition, a multivariable cox regression with all items in FORECAST was performed with stepwise backward elimination.

The discriminative ability was determined for each score by the concordance-statistics (C-statistics) using Uno’s method on censored survival data [18], integrated discrimination improvement (IDI) and net reclassification improvement (NRI) without categories. C-statistics express the probability that 1 individual who experiences an event has a higher predicted probability of an event than an individual who does not experience an event. Larger C-statistics values indicate better discrimination. A value of 0.5 indicates random prediction, and a value of 1 signifies perfect prediction. The Wald test was used to compare the C-statistics of different scores and is illustrated by time-dependent receiver operating characteristic (ROC) area under the curve (AUC) estimated by the inverse probability of censoring weighting [19].

IDI quantifies the increase in separation using differences between means of model-based probabilities for events and non-events [20]. NRI quantifies the amount of correct change in model-based probabilities expressed as the net proportion of events with increased model-based probability (NRI_e) and the net proportion of non-events with decreased model-based probability (NRI_ne). NRI_e and NRI_ne can each be interpreted as the percentage of events classified with higher probability and non-events classified with lower probability, respectively [20].

Decision curve analysis was performed after 1 year (PMID 17099194) using software package at https://www.mskcc.org/departments/epidemiology-biostatistics/biostatistics/decision-curve-analysis (8 December 2020).

Statistical significance was defined as P-values ≤0.05, and all P-values were two-sided.

Statistical analyses were performed using SAS statistic version 9.4 (SAS Institute Inc., Cary, NC, USA) and R version 3.5.1 (Vienna, Austria).

We adhere to the Strengthening the Reporting of Observational Studies in Epidemiology statement.

Ethical statement

The detailed protocol and criteria were clearly defined prior to study initiation and approved by the Regional Ethical Review Board at the University of Copenhagen, reference number H-16020318, 13 April 2015. Informed consent was obtained before assessment in all patients. The study is registered in Clinicaltrials.gov (Identifier: NCT02992587).

RESULTS

Baseline and clinical outcomes

The study included 604 patients, of whom 21% were women, and the median age was 73 years (range: 65–90 years). Preoperative characteristics are shown in Table 1. Fifty-six percentage underwent coronary artery bypass graft, 29% valve replacement and 15% combined procedures. The median EuroSCORE II was 2.0 [interquartile range (IR) 1.2–3.3], the median STS score was 1.3 (IR 0.8–2.1), the median CAF score was 7.0 (IR 5.0–10.0) and the median FORECAST was 4.0 (IR 3.0–6.0). The median follow-up period was 3.4 years (IR 2.9–3.7), with a mortality of 13% (79) (Fig. 1). The cause of death was cardiac related in 19 patients, respiratory failure in 5 patients, intestinal ischaemia in 4 patients, infection in 8 patients, cerebral related in 3 patients, cancer in 15 patients and unknown in 25 patients.

Predictive ability

In a univariable Cox proportional hazards regression, EuroSCORE II, STS score, CAF score and the FORECAST were significant predictors of all-cause mortality (Table 2). In a multivariable Cox proportional hazards regression, CAF score and FORECAST were independent predictors when adjusting for both EuroSCORE II and STS score (Table 2). Discriminative ability measured by the C-statistics found the STS score to have the highest value of 0.70, followed by the CAF score with a value of 0.69 and FORECAST with a value of 0.68; which are all slightly higher than the EuroSCORE II with a value of 0.64. None of the risk scores’ C-statistics were significantly different from each other. Time-dependent AUC of ROC curves for each risk score is illustrated in Supplementary Material, Fig. S1.

Additive value

The addition of the CAF score or FORECAST to either the EuroSCORE II or STS score increased the C-statistics, but not significantly (Table 3). The highest value, 0.73 (95% CI: 0.67 to 0.78), was seen when the CAF score was added to the STS score. The largest numerical increase in C-statistics by 0.053 was observed when the CAF score was added to the EuroSCORE II.

The time-dependent AUC of ROC was stable throughout the study when adding the CAF score or FORECAST to either the EuroSCORE II or the STS score. With an AUC that was greater for both the EuroSCORE II and the STS score when CAF score or FORECAST was added (Supplementary Material, Figs. S2 and S3).

An increase in the performance of the model, assessed as IDI and NRI, was observed by adding CAF score or FORECAST to either the EuroSCORE II or the STS score (Table 4).

IDI improved between 0.01 and 0.05 by adding the CAF score or FORECAST to both EuroSCORE II and STS score, respectively. The increase was significant for all scores at 2 and 3 years.

Significant improvement of NRI was seen when CAF score and FORECAST were added to either EuroSCORE II or STS score for the model-based probability of non-events but did not significantly change the model-based probability of events (Table 4).

We performed a decision curve analysis at 1 year (Supplementary Material, Figs. S4–S7) [21]. For models with STS score, the addition of frailty scores increased net benefit from a threshold probability of 2% up to 28% with decreased benefit above this point. For EuroScore II models, the addition of frailty scores added benefit consistently across threshold probabilities of 2–23%.

With the similar improvement of the existing scores when adding either CAF score or the simplified FORECAST, the additive effect on mortality prediction of the 5 items of the FORECAST was further analysed. In a multivariable Cox proportional hazards regression model, chair rise and serum creatinine >110 µmol/l were significant risk factors independent of EuroSCORE II. As EuroSCORE II already includes creatinine clearance, serum creatinine was not included in the subsequent analyses. For the STS score, only chair rise was a significant risk factor, which also showed significant interaction with the STS score. By adding chair rise to the existing scores, higher C-statistics were seen, but this difference was not significant. The C-statistics value changed from 0.64 (95% CI: 0.58–0.70) to 0.68 (95% CI: 0.62–0.74; P = 0.301) when added to EuroSCORE II, and from 0.70 (95% CI: 0.64–0.76) to 0.73 (95% CI: 0.67–0.79; P = 0.109) when added to the STS score. Both IDI and NRI improvements were similar when CAF score or FORECAST was added to either the EuroSCORE II or the STS score (Table 4).

DISCUSSION

In this study, we have demonstrated that CAF score and FORECAST can predict increased risk of death as well as the existing scores EuroSCORE II and STS score. Furthermore, CAF score and FORECAST added to the existing risk scores increase prediction with improved IDI and NRI_ne performance for the prediction of mortality in patients undergoing cardiac surgery.

Frailty is widely accepted as a predictor of mortality in patients undergoing cardiac surgery [8] and some studies have reported an additive effect of frailty to the existing risk scores for patients undergoing cardiac surgery [9–14, 22].

One of the first studies to investigate the additive effect of frailty to the existing risk scores measured frailty by gait speed [10]. Based on 131 patients undergoing cardiac surgery, gait speed had an additional effect on STS score with increasing AUC from 0.70 (95% CI: 0.60–0.80) to 0.74 (95% CI: 0.64–0.84) and improved IDI by 0.05 in predicting 30-day mortality.

In another study, including 254 patients undergoing cardiac surgery, frailty assessment was based on the Edmonton Frail Scale [12]. Edmonton Frail Scale showed additional value to EuroSCORE II for the prediction of 30-day mortality by significant higher log-likelihood ratio test (P = 0.04) and improved IDI by 0.03 (P = 0.01).

Both studies showed similar results as ours with an additional improvement of the existing risk scores. However, in contrast to the above-mentioned studies, but similar to ours, another study by Afilalo et al. [13] compared the additional effect of frailty scores on both the STS score and the EuroSCORE on the same population. These authors included 152 patients undergoing cardiac surgery and measured frailty by 4 different scores (5-item Cardiovascular Health Study, 7-item CHS, 4-item MacArthur Study of Successful Ageing and gait speed). These authors found STS score superior to the EuroSCORE. This is in line with our results, with a numerical lower C-statistics value for EuroSCORE II both alone and when CAF score or FORECAST was added, although not statistically significant. A non-significant C-statistics finding does not necessarily mean that they do not provide a better model [23, 24] as it can be difficult to reach significance if the models (EuroSCORE II and STS score) are good predictors in advance.

The 3 studies above report the prediction of 30-day mortality. From a patient’s perspective, this is a very short time span. Expanding the existing risk score to predict long-term mortality risk seems reasonable. Also, frail patients undergoing cardiac surgery return to the preoperative baseline status in terms of physical capacity and quality of life up to 1 year after surgery, following a U-shaped curve [25, 26]. This strengthens the importance of long-term prediction and thereby the importance of improving the EuroSCORE II and STS score since both these scores are validated only for 30-day mortality. In our study, we found that the CAF score and the FORECAST are able to improve both the risk scores for long-term prediction up to 3 years after cardiac surgery.

In a study of 188 patients undergoing cardiac surgery, frailty was measured by 3 different scores (the Modified Fried Criteria, the Short Physical Performance Battery and the CFS score). When added to EuroSCORE II, all 3 frailty test improved the prediction of 1-year poor functional survival, with improved IDI up to 0.067 and category-free NRI up to 0.596 [14].

To determine the clinical usefulness of the added frailty score, we performed a decision curve analysis after 1 year. The analysis showed, overall, a net benefit of adding frailty scores but with modest increase in net benefit. The analysis depends on the, a priory, chosen threshold probability of 1-year mortality, which results in a decision whether or not to perform operation. In this setting, the clinical cost of performing a frailty assessment (as there should be no harm in doing so) has to be weighed against the found maximum increase in net benefit of ∼0.5%. Furthermore, an informed decision on whether or not to perform surgery should include an estimation of the risk without surgery—information which is difficult to assess in the clinical setting.

The Essential Frailty Toolset is a widely mentioned simplified score for frailty assessment in recent literature, with the highest published AUC of 0.78 when added to the STS score [27], and Essential Frailty Toolset is a measure of chair rise, cognitive impairment, haemoglobin and serum albumin. CAF score is a multidimensional test and takes up to 20 min, which is a disadvantage. Therefore, the FORECAST was introduced, which showed a similar predictive and additive effect. However, in our study, we also found that the single item of chair rise is an independent risk factor for the prediction of mortality and has a similar additional value to the existing risk scores, increasing the C-statistics to 0.73. This is an interesting finding and of particular interest since it shows as high discrimination as when the CAF score was added and is easily and quickly performed.

Chair rise as a standalone test for the prediction of mortality in patients undergoing cardiac surgery is strengthened by a previous study, which included 228 patients undergoing aortic valve replacement, of whom 91 underwent surgical aortic valve replacement. Frailty was assessed through 5 frailty scores (fatigue, resistance, ambulation, illness and loss of weight scale, CFS, grip strength, gait speed and chair rise). The authors found that chair rise had the highest C-statistics of all scores (C-statistics = 0.76) [28].

Limitations

There are some limitations to our study. First, this is a single-centre study with a risk of selection bias. Second, some patients were preoperatively found to be frail and thereby not suitable for surgery. These patients were not included. Third, we use the CAF score and FORECAST to define frailty, but other frailty assessment scores might further improve the existing score. Fourth, patients in the need of emergency surgery were excluded. Fifth, all patients are seen by the Heart team, and the number of patients considered unsuitable for cardiac surgery is unknown. A further limitation of the study is the low number of early events.

CONCLUSION

Frailty is an important factor when estimating preoperative patient risk profiles. CAF score and FORECAST are good at identifying high-risk patients. Estimation of risk prediction when the CAF score, FORECAST or chair rise are added showed supplemental value to the existing risk scores, EuroSCORE II and STS score. Therefore, we advocate for the use of frailty assessment in clinical practice. Simplified scores, such as the FORECAST, are strongly advocated for clinical reasons. Implementation of the chair rise test alone might be sufficient to receive a realistic assessment for the existence of frailty. Further research of these frailty assessment methods is now needed.

SUPPLEMENTARY MATERIAL

Supplementary material is available at EJCTS online.

Funding

The project was sponsored by the Department of Cardiothoracic Surgery, Rigshospitalet, Copenhagen, University Hospital, Denmark.

Conflict of interest: none declared.

Author contributions

Caroline Bäck: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Software; Validation; Visualization; Writing—original draft; Writing—review & editing. Mads Hornum: Conceptualization; Formal analysis; Methodology; Resources; Supervision; Writing—original draft; Writing—review & editing. Morten Buus Jørgensen: Formal analysis; Methodology; Resources; Writing—review & editing. Ulver Spangsberg Lorenzen: Formal analysis; Investigation; Methodology; Resources; Writing—review & editing. Peter Skov Olsen: Conceptualization; Formal analysis; Funding acquisition; Investigation; Methodology; Resources; Supervision; Writing—original draft; Writing—review & editing. Christian H. Møller: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Resources; Software; Supervision; Writing—original draft; Writing—review & editing.

Reviewer information

European Journal of Cardio-Thoracic Surgery thanks Kerem M. Vural and the other, anonymous reviewer(s) for their contribution to the peer review process of this article.

REFERENCES

ABBREVIATIONS

 
• AUC

  Area under the curve

 
• CAF

  Comprehensive assessment of frailty

 
• CFS

  Clinical frailty scale

 
• CI

  Confidence interval

 
• C-statistics

  Concordance-statistic

 
• EuroSCORE

  European System for Cardiac Operative Risk Evaluation

 
• FORECAST

  Frailty predicts death one year after elective cardiac surgery test

 
• IDI

  Integrated discrimination improvement

 
• IR

  Interquartile range

 
• NRI

  Net reclassification improvement

 
• ROC

  Receiver operating characteristic

 
• STS

  Society of Thoracic Surgeons