Model of lung cancer surgery risk derived from a Japanese nationwide web-based database of 78 594 patients during 2014–2015^†

Abstract

INTRODUCTION

Lung cancer is a leading cause of death worldwide. Surgery remains a mainstay for complete cure. Because of the large number of elderly people in Japan, lung cancer patients frequently have multiple comorbidities, which increase mortality and morbidity risks. Risk-adjusted outcome analysis is demanding when used to assess preoperative risk, monitor surgical performance and implement measures that improve care.

In 2011, the National Clinical Database (NCD) of Japan adopted an annual web-based data collection system. The NCD is a nationwide system that links data collection to the first level of surgical specialization in the Japanese Surgical Board Certification System. In 2014, data on 1.6 million surgical procedures from more than 4000 hospitals were collected [1]. On the basis of the existing NCD system, an NCD specializing in general thoracic surgery was launched in 2014. The data registration system and information recorded are described in detail in our previous report [2]. The NCD for general thoracic surgery is part of the second level of specialization in general thoracic surgery and the accreditation system for educational institutions. In total, 80 095 lung cancer operations were registered in the 2014 and 2015 data sets. More than 95% of all pulmonary resections for lung cancer registered with the Regional Bureau of Health and Welfare in Japan were accounted for in the NCD [3].

In this study, we used the NCD for general thoracic surgery to develop and validate a model estimating individualized risk for patients undergoing pulmonary resection for lung cancer.

MATERIALS AND METHODS

Patient population

The study population for the current analysis was derived from 2 annual data sets (2014 and 2015) that included information on persons who underwent surgical resection for primary lung cancer (at 797 surgical units in 2014 and 814 surgical units in 2015). Surgical approach was categorized as thoracotomy or as a minimally invasive approach, including complete video-assisted thoracoscopic surgery and mini-thoracotomy with a wound length of 8 cm or less. The variable ‘surgical approach’ was excluded in the development of risk models, because decisions regarding surgical approach were biased by variability in patient selection at the different centres. The surgical procedures included wedge resection, segmentectomy, lobectomy, sleeve lobectomy, bilobectomy and pneumonectomy. Sleeve lobectomy or bilobectomy was entered as a simple variable (lobectomy or bilobectomy) in the analysis, because decisions regarding selection of sleeve lobectomy varied among centres and because few patients underwent bilobectomy (n = 143 in 2014 and n = 134 in 2015). Nodal dissection was categorized as hilar, lobe-specific mediastinal or systematic, as shown in Table 1.

Patients were excluded if they had undergone procedures with no curative intent (n = 363 in 2014 and n = 359 in 2015), extrapleural pneumonectomy, completion pneumonectomy, emergency surgery or combined procedures for both lungs or if they had been transported by ambulance. Of the 39 029 patients in 2014 and 41 016 patients in 2015 who were eligible for analysis, 1501 (n = 752 in 2014 and n = 749 in 2015) were excluded from the analysis. The development data set for risk models included 38 277 patients entering in 2014, and the validation data set included 40 317 patients entering in 2015.

Outcome measures

The primary outcome measures were operative mortality and the composite outcome of mortality/major morbidity. Operative mortality included patients who died within 30 days after surgery, regardless of hospitalization status, and those who died within the index hospitalization, even if death occurred after transfer to another hospital.

Major morbidity was defined in accordance with the Society of Thoracic Surgeons (STS) risk models as shown in Table 2 [4, 5].

Statistical analysis

For the multivariate logistic regression analysis, a risk model set, registered in 2014 (n = 38 277), and a validation set, registered in 2015 (n = 40 317), were created to estimate associations of patient baseline characteristics with the primary outcome measures of operative mortality and composite mortality/major morbidity. The variables entered in the model were selected with the χ² test, for categorical covariates, and the unpaired t-test, for continuous covariates. All variables that were significant at P < 0.05 and were present in at least 0.5% of the sample were included in the multivariate stepwise logistic regression analysis of both outcomes. Missing or inconsistent values for age or spirography were substituted with the most frequent categories. Model discrimination was assessed by examining the area under the receiver–operating characteristics curve (C-statistic). Model variation was analysed using the annual data registered in 2015. Analyses were performed with the IBM SPSS Statistics software package (version 23; IBM Corp., Armonk, NY, USA).

RESULTS

Risk profile of study population

The number of patients who underwent lung cancer surgery was 38 277 in the risk model set (1 January 2014 through 31 December 2014) and 40 317 in the validation set (1 January 2015 through 31 December 2015). The baseline characteristics of these patient groups were similar (Table 1).

Outcomes

The most frequent cause of major morbidity was respiratory failure after pneumonia and atrial arrhythmia (Table 2). In the risk model set, there were 315 (0.8%) operative deaths, and major morbidity was noted in 2134 patients (5.6%). In the validation set, there were 309 (0.8%) operative deaths, and major morbidity was noted in 2261 patients (5.6%) (Table 3). Mortality and major morbidity rates were high in patients who underwent pneumonectomy and not influenced by hospital volume (Table 4). More than half of the dead patients died from respiratory-related death, including acute exacerbation of interstitial pneumonia, pneumonia and other respiratory failure. The outcomes of the 2 data sets were almost identical.

Model result

Multivariate risk models were developed, and the final logistic model, with odd ratios and 95% confidence intervals (CIs), is presented in Table 5, which shows the associations of patient baseline characteristics with the outcome measures of mortality and mortality/major morbidity. Nineteen variables were associated with mortality, and 25 variables were associated with mortality/morbidity.

To evaluate model performance, we used the concordance C-index (a measure of model discrimination), which is the area under the receiver–operating characteristics curve. The C-indices were 0.854 (95% CI, 0.835–0.874; P < 0.001) for mortality and 0.718 (95% CI, 0.708–0.729; P < 0.001) for mortality/major morbidity. The C-indices in the validation data sets for these 2 models were 0.849 (95% CI, 0.830–0.868; P < 0.001) and 0.723 (95% CI, 0.713–0.733; P < 0.001), respectively.

DISCUSSION

Surgery is a promising treatment for patients with early stage lung cancer, even though some postoperative complications are unavoidable. Risk ratios for postoperative complications are affected by patient demographics, oncologic factors such as histology and staging, type of surgical procedure and surgeon performance. The STS [5], the US National Cancer Database [6], the European Society of Thoracic Surgeons (ESTS) [7] and institutions in other countries [8] have developed risk models for lung cancer surgery, to assess quality measures for surgeon performance and preoperative decision-making.

Obstacles to establishing optimal risk models

Several limitations in establishing ideal risk models have been described [9], as follows:

(i) Risk models should be based on a large database covering as many operations as possible. The criteria used to select patients for surgery may vary by centre, which would have affected the model. A clinical database will not yield an accurate risk model unless there is a high participation rate in data entry [2]. Different investigators evaluating the same predictors by means of regression analysis might obtain heterogeneous results because of sample biases at the time of the traditional training-and-test method for model building. To maintain statistical reliability and reproducibility, bootstrap analysis is recommended [9]. In addition, the use of different samples is recommended for model validation processes. Data registered in the following year, 2015, were used for validation, to show the reproducibility and reliability of our risk model.

(ii) The use of morbidity as an outcome is problematic. Although it is defined in the manual for the case report form in the NCD registration system [2], morbidity is subject to entry error or under-reporting. In Japan, data managers receive instruction on the correct registration of postoperative complications, at the NCD seminar at the annual meeting of the Japanese Association for Chest Surgery.

(iii) Important variables, such as a body mass index >35, are not included in the present model because they are infrequent in the population [10]. The NCD does not include race or presence of peripheral vascular disease among the preoperative variables. In addition, deep vein thrombosis and sepsis are not included among postoperative complications. However, these complications are rare in Japan. The STS and ESTS databases do not include interstitial pneumonia or liver cirrhosis as variables [5, 7], even though our model identified them as significant risk factors for lung cancer surgery. These variations in risk models, which are related to regional differences in data collection, should be carefully reviewed. A worldwide clinical database, with the same variables included in all countries, is desirable [11].

(iv) Previously reported risk models by the STS and ESTS defined operative mortality as death during the index hospitalization for surgery or within 30 days of the procedure. In our database, operative mortality classified death after transfer to another hospital over 30 days after surgery as an in-hospital death. The universal health care system in Japan allows patients with serious comorbidities or postoperative complications to be transferred to another hospital. Mortality at 90 days after thoracic surgery is twice that at 30 days [12]. In the present study, the value for 30-day mortality plus in-hospital mortality is double that for 30-day mortality and thus is likely to be approximate 90-day mortality.

(v) Data quality affects risk model analysis. Databases should be audited regularly to maintain the quality of information; however, the audit of even a small fraction of a database requires substantial effort. To reduce the burden of ensuring data quality, a web audit system was developed for the Japan NCD system. Surgeons provide anonymous operative notes of patients randomly selected by the NCD, the number of which is equivalent to approximately 0.5% of registered cases. These notes are submitted to the Japanese Board of General Thoracic Surgery at the time of application for board certification for general thoracic surgery. A committee authorized by the NCD determines inter-rater reliability between these samples and Internet-based data from the NCD. The results indicate that the correctness of data on age, gender, procedure, disease, operative time, blood loss and participating surgeons was greater than 94% [2]. Starting in 2017, the web audit will encompass other variables related to patient demographics and outcomes, which will be evaluated using hospitalization summaries randomly selected by the NCD office.

The aim of this study was to use the comprehensive NCD on general thoracic surgery to develop a risk model for lung cancer patients undergoing pulmonary resection. The 30-day and the rate of the composite outcome morbidity/major morbidity were lower than those for the STS and ESTS databases, perhaps because of differences in clinical characteristics (such as body mass index and comorbidities), clinical staging and type of surgery, i.e. the so-called ‘cherry-picking’ problem. Patient demographics in the NCD differed greatly from those in the STS and ESTS database-based risk model of 2016, specifically the distributions of patients undergoing induction treatment, thoracotomy and pneumonectomy (Table 6) [5, 7]. Despite differences in operative morbidity and mortality between the STS risk model and our NCD model, there were several shared risk factors for operative mortality and morbidity. If input items were standardized, a large clinical database could overcome problems related to regional disparities.

Older age and being male were predictors in both risk models [5, 6]. Male sex has consistently been identified as a risk factor in other models of lung cancer surgery risk. The fact that advancing age had adverse effects on mortality and morbidity in the present patients younger than 80 years but not in those aged 80 years or older is likely attributable to selection bias. Physical performance might be similar for surgical candidates in these age groups. Interstitial pneumonia and comorbidities such as haemodialysis and liver cirrhosis were significant risk factors. Interstitial pneumonia is diagnosed on the basis of a radiologic finding of a fibrotic shadow with traction bronchiolectasis in bilateral basal segments, regardless of respiratory symptoms and diffusing capacity of the lung carbon monoxide. Affected patients are susceptible to lethal postoperative respiratory failure from acute exacerbation of interstitial pneumonia [13]. The diffusing capacity can be accepted as important predictors of operative mortality and morbidity after lung cancer resection [7]; therefore, these data have been documented since 2017.

Respiratory acidosis caused by respiratory failure is fatal for patients undergoing maintenance haemodialysis who cannot compensate for respiratory acidosis and hyperkalaemia by metabolic alkalosis [14]. Child–Pugh Class B/C liver cirrhosis can cause malnutrition, prolonged pleural discharge and bleeding and may thus be a risk factor [15].

A tumour diameter >3 cm and a clinical stage of 2 or higher are important variables relating to extensive resection and are likely to be important risk factors. Evidence from numerous studies indicates that minimally invasive surgery, including video-assisted thoracoscopic surgery, has favourable effects on mortality and morbidity. However, candidates for minimally invasive surgery may be more likely to have early lung cancer. Thus, the surgical approach should not be included in the risk calculation because of likely selection bias and institutional bias. Propensity-matched analysis is required in order to clarify risk with respect to the use of a minimally invasive approach [16]. In addition, because of the increasing number of patients with less invasive adenocarcinoma, the variable non-adenocarcinoma was identified as a risk factor.

Pneumonectomy resulting in cardiopulmonary dysfunction and bronchopleural fistula is an important risk factor in lung cancer surgery [17]. In our model, other risk factors were right lower lobectomy—which can result in the presence of a large pleural dead space in the thoracic cavity, where bronchopleural fistulae might develop [18]—and chest wall resections other than resection of the first rib, which may cause postoperative respiratory failure [19].

Our analysis showed that risk models based on the NCD 2014 could be validated with the NCD 2015. Validation analysis confirmed the feasibility of the risk model. However, patient demographics, lung cancer oncology and treatment strategy, among other factors, will continue to change as societies age and medical science advances. Furthermore, the 8th edition of the tumour, node and metastasis classification for lung cancer was published by the International Association for the Study of Lung Cancer. If adverse effects resulting in operative mortality and morbidity are changing, the risk models for lung cancer surgery will need to be reviewed, particularly in rapidly changing societies. Our NCD data collection system will continue to enable data managers to respond to changes in data input, as the system is able to annually update risk models for lung cancer surgery.

ACKNOWLEDGEMENTS

The authors thank all the data managers and hospitals participating in this NCD project, for their considerable efforts in data entry.

Funding

This work was supported in part by a Health Labour Sciences Research Grant (201313055 C).

Conflict of interest: none declared.

REFERENCES

APPENDIX. CONFERENCE DISCUSSION

Dr W. Weder(Zurich, Switzerland): Are you prospectively validating the quality of your risk model, with other words, are you looking what’s happening to these patients you have predicted a certain risk during a year?

Dr Endo: We performed just a retrospective analysis, not prospective.

Dr Weder: Yes, I understand, but are you now prospectively evaluating the quality of the risk model?

Dr Endo: I hope so.

Dr Weder: And second question: did you look at the hospital-specific mortality, especially at the risk of, was it dependent on the size? I have not seen it in the data.

Dr Endo: In our data, we cannot find the difference between hospitals depending on volumes of surgery.

Dr K. Naunheim(StLouis, MO, USA): In the USA, we are also struggling with quality improvement and feedback, and it is hard to know what to do with this information. Our own databases, like yours, give the observed to expected ratio and there is a statistical analysis that will tell you, just because you are 1.1 or 1.2 does not mean you are outside the realm of good surgery. Sometimes that could be just a statistical blip. I want to know whether you have some method for statistically analysing the ODE. Do you feed that information back to the surgeon, is that something they get and if they are outside the bounds of acceptable practice, what sort of remediation, what sort of help can you give them to try to bring them back to where they should be, because it is not that they are bad people, they are necessarily bad surgeons. Almost everybody who is a thoracic surgeon can be made better. The whole idea of quality improvement is not punitive, is not to call people names or kick them out of the profession, we need to re-educate them and bring them back in, so I do not know whether you have any programmes to help re-educate surgeons.

Dr Endo: In our field of thoracic surgery, we do not perform such audit systems, to improve the surgical performance. But in the field of cardiovascular surgeries, the teams can advise the improvement of the hospital performance, actually, in the last year. I also hope that if we find some bad hospital performance I can recommend some improvement.

Author notes