External validation of the Eurolung risk models: a necessary analysis to audit their reliability

Eurolung models have been published by the European Society of Thoracic Surgeons Database Committee to adjust morbidity and mortality outcomes with the purpose to be used among other quality indicators as part of the Composite Performance Score [1, 2]. This score incorporates outcome and processes end-points in a single score to assess the eligibility of the centres participating in the ESTS Database to the Institutional Accreditation Process [3–5].

The most recent version of the Eurolung is the latest evolution of a series of risk models published by the ESTS Database Committee over nearly 2 decades. It has been developed from a population of over 80 000 patients submitted to anatomic lung resections from 2007 through 2018 [2].

The study from the Spanish Group of Video-Assisted Thoracic Surgery (GEVATS) [6] aimed at validating Eurolung models in their cohort of patients. This group of patients was operated more recently (December 2016–March 2018) and was 29-fold smaller than the original ESTS cohort used to develop the models.

The authors found that Eurolung2 (risk-adjusted mortality model) was well calibrated and had good discrimination in their population, whilst the Eurolung1 (risk-adjusted morbidity model) performed less well.

The good external performance of the mortality model is reassuring, whilst the underperformance of the morbidity model raises some concerns. First of all, morbidity is known to be affected by problems of definition, misclassification and recording. On the other hand, one should expect that similar problems have occurred in both databases. The overall morbidity rate in the GEVATS database is lower (12%) compared to the one in the ESTS database (16%). This may be explained by the much higher proportion of minimally invasive cases registered in the Spanish database (54% vs 26%). It is now well known that minimally invasive surgery has been associated with improved short-term outcomes [7, 8] Although the surgical approach [Video-assisted thoracoscopic surgery (VATS) versus open] is accounted for in the Eurolung model, its impact could have been underestimated (in terms of regression coefficient weight) due to under-representation of the minimally invasive operations in the ESTS cohort. This highlights the importance of regularly updating the models to better reflect the current surgical case mix.

The study from de Antonio et al. [6]. represents an important contribution which serves the purpose of quality checking our current European models. It would be desirable that more independent investigations will follow and repeat the same external validation process. Analyses such as the one from the GEVATS are important for the ESTS community as we can react upon it and take the necessary adjustments to refine the models accordingly.

For instance, this study may prompt a future recalibration of the Eurolung in a most recent cohort of patients which will be more representative of our current surgical case mix. As a practical example, the proportion of lobectomies performed through minimally invasive surgery has increased from 10% in the first 6 years of the ESTS database (2007–2013) to 47% in the most recent period (2014–2021) questioning the representativeness of the early cohort and whether it would be more appropriate to limit the model development from the most recent group of patients. Similarly, the rate of pneumonectomies nearly halved from the earliest period in the ESTS database from 11% to 6.5% in the late phase (214–2021). Unfortunately, there is no information on the breakdown of resections in the GEVATS series and it is therefore difficult to make any comparison in that regard.

Finally, we want to stress the concept that population-based models should not be used on the individual patient with the purpose to select patients for surgery. They only should be used as additional information aid tools to inform the discussion at the Multidisciplinary team (MDT) meetings and during the shared decision-making process, when presented and discussed with the patients in the appropriate way. Their main role remains to be an instrument to audit performance, ideally in combination with multiple outcome end points and processes measures [i.e. Composite Outcome Index (COI) or Composite Performance Score (CPS)] [4, 5, 9].

Dr. de Antonio et al. [6]. should be commended for having tested Eurolung models in their independent population and we look forward to more independent similar analyses to contribute to the refinement of the ESTS quality measures.

REFERENCES