Time for clinicians to revisit their perspectives on C-statistic

This commentary refers to 'Feasibility of using deep learning to detect coronary artery disease based on facial photo' by S. Lin et al., 2020;41:4400–4411.

Lin et al.¹ did a weighty contribution to improve our ability to predict the presence of coronary artery disease (CAD) in those undergoing coronary angiography (CAG) or coronary computed tomography angiography (CCTA). They developed a deep learning algorithm based on facial features and demonstrated that its prediction power is greater than the Diamond–Forrester model and the CAD consortium clinical score based on C-statistic.¹ We believe that it is a worthy attempt to aid us in enrolling our patients for CAG or CCTA. A major diagnostic challenge is finding a valid and reliable means of distinguishing patients with normal or near-normal coronary arteries from those with obstructive CAD based on clinical characteristics and non-invasive evaluations. If a reliable tool was available, cardiologists could reduce the number of unnecessary CAG procedures, which are associated with a small but definite risk to the patient, healthcare costs, and waste of medical resources.² In this commentary, we are going to challenge the use of C-statistic and suggest some modern indices for comparing prediction models.

The harm-to-benefit ratio equals to the odds of the clinically acceptable cut-off for decision-making. Lin et al.¹ provided C-statistic for the Diamond–Forrester model and the CAD consortium clinical score, while they did not mention any cut-off, i.e. 5%, 30%, 40%, or 70%. Taking a cut-off into account and calculation of the NB index confers a more clinical weight to the comparison.⁴ At last, we should emphasize that we do not question the valid attempt of Lin et al.  ¹; nevertheless, we strongly recommend to use a clinically acceptable cut-off for making decisions and employing the NB index in future studies.⁴

Conflict of interest: none declared.

References