Reply to Nezic

We thank Dr Nezic for his comments and interest in our recently published study to assess the external validation of a model to risk stratify patients for developing postoperative atrial fibrillation (POAF) after thoracic surgery [1, 2].

Ultimately, the value of a prediction model relates to its ability to influence clinical decision-making [3]. As we discussed in our paper, we are in need of a method for identifying the cohort of patients who would benefit the most from POAF prophylaxis in thoracic surgery. From our review of the literature, no model has been validated that fits this purpose; however, we did find several models in the literature for stratifying patients’ risk of developing POAF [2]. Given the existing models, we chose to select the strongest prediction model suitable for prophylaxis (i.e. variables were obtainable preoperatively) and assessed its performance in our cohort of patients [4]. Our aim is to utilize this model for future targeted prophylaxis.

In general, we agree with the importance of assessing both discrimination and calibration of prediction models at the time of model development and that these tests would be important in the evaluation of the external validity of a model, which is predicting a specified risk of an outcome [5]. However, the POAF prediction model we discussed does not predict a specific incidence of POAF. Rather, it identifies the cohort of patients at increased risk of developing POAF without giving an absolute incidence. Therefore, the statistical tests to evaluate calibration are not applicable in this context. We assessed the model through a graphical representation of the risk score (x-axis) against the observed proportion with POAF (y-axis). Because we were not predicting a specific POAF incidence, we were unable to calculate the goodness-of-fit. Instead, we looked at the general trend and clinical significance of the difference in POAF incidence between the final low versus high-risk cohorts. As for the discrimination, we found the receiver operating characteristic demonstrated a positive trend and the area under the receiver operating characteristic curve was 0.62. We did not include this assessment in our publication because it does not contribute to the primary objective, namely identifying a higher risk cohort suitable for targeted prophylaxis.

In summary, while prediction in and of itself offers value to clinicians, actionable risk stratification tied to prophylaxis has the capacity to directly impact care. By identifying individuals with a higher risk of POAF, it is hoped that targeted prophylactic therapy will be safer and more effective at reducing POAF. This aspiration merits further assessment with formal rigorous clinical investigation.

REFERENCES