Diagnostic accuracy of a high-sensitivity troponin I assay and external validation of 0/3 h rule out strategies

Abstract

Aims 

The timely diagnosis and exclusion of acute coronary syndromes in the Emergency Department (ED) remains a challenge. This study aims to evaluate the diagnostic accuracy of a high-sensitivity cardiac troponin I assay (Siemens TNIH) on serial sampling for ED patients as standalone test and in rule-out algorithms as recommendations remain assay specific.

Methods and results 

This secondary analysis from a prospective diagnostic accuracy study at 14 centres included ED patients presenting with chest pain of suspected cardiac nature. Serum drawn on arrival and 3 h later was batchtested for TNIH. The target condition was an adjudicated diagnosis of acute myocardial infarction (AMI). We evaluated the diagnostic accuracy of absolute and relative delta criteria and four rule-out strategies. Of 802 included patients, 13.8% had AMI. Absolute delta criteria had superior accuracy to relative criteria (C-statistic 0.94 vs. 0.76, P < 0.001). However, no delta criteria achieved >95.5% sensitivity for AMI when used alone. Ruling out AMI with TNIH below the 99th percentile at 0 and 3 h had 88.3% (95% confidence interval 80.8–93.6%) sensitivity. The adapted European Society of Cardiology (ESC) 0/2 h algorithm had higher sensitivity (98.2%) than both High-STEACS (93.7%, P = 0.03) and the ESC 0/3 h algorithm (79.3%, P < 0.001). These pathways ruled out 63%, 74%, and 88% patients, respectively.

Conclusion 

With serial sampling over 3 h, the Siemens TNIH assay should be used with a validated algorithm incorporating bespoke cut-offs and absolute delta criteria. In our analysis, the adapted ESC 0/2 h algorithm had greatest sensitivity. ‘Ruling out’ AMI using the 99th percentile of the assay cannot be recommended.

Open in new tabDownload slide

Introduction

The rapid diagnosis and exclusion of acute coronary syndromes (ACSs) in patients who present to the Emergency Department (ED) with chest pain remains a challenge to emergency physicians worldwide. This heterogeneous patient group makes up 25% of acute medical admissions in the UK, yet only 8–10% of the patients have a diagnosis of ACS. The low prevalence of ACS despite the high number of presentations presents a significant demand on finite health care resources.^1,2

Currently, serial cardiac troponin testing remains the standard of care to rule out acute myocardial infarction (AMI). However, the optimal algorithm to rule out AMI should be defined for every assay.

The European Society of Cardiology (ESC) 2015 guideline supported the use of early serial measurements of cardiac troponin using a high-sensitivity cardiac troponin (hs-cTn) assay, with samples drawn on arrival in the ED and 3 h later (the ‘ESC 0/3 h pathway’).^3,4 The ESC 0/3 h pathway uses the 99th percentile upper reference limit as a cut-off for both ruling in and ruling out AMI, in combination with delta changes. A similar approach was endorsed by the National Institute for Health and Care Excellence (NICE), supporting 0/3 h hs-cTn measurements for AMI rule out using the 99th percentile cut-off.^5,6

The ESC 2015 guidelines have since been superceeded by the ESC 2020 guidelines that have moved to recommend a 0/1 or 0/2 h algorithm as rapid rule in and rule out strategies with troponin assay-specific cut-offs for optimal performance.⁷ While the 0/1 h algorithm is deemed the preferred approach, delivering blood sampling in such short intervals can proof difficult in busy EDs potentially making a 0/2 h sampling more practical as well as many places worldwide still using the now downgraded conversative 0/3 h ESC 2015 approach.

Cut-offs for using the Siemens ADVIA Centaur High-Sensitivity Troponin I assay (Siemens TNIH assay) with the ESC 2020 algorithms were derived and validated in the Advantageous Predictors of Acute Coronary Syndromes Evaluation (APACE) study where the 0/2 h algorithm could rule out AMI in ∼55% of patients with high negative predictive value (NPV).⁸

Similar to the ESC 2020 approach, Chapman et al. derived and validated the High-STEACS (high-sensitivity troponin in the evaluation of patients with ACS) pathway using optimized hs-cTn cut-offs set below the 99th percentile of the assay to improve risk stratification. In that study, the High-STEACS pathway ruled out more AMI patients than the ESC 0/3 h pathway, while having a five-fold lower miss rate of index AMI.⁹

All three algorithms use a combination of the concept of absolute troponin delta (AΔ = |TNIH concentration_T=3h − TNIH concentration_T=0h|) with the High-STEACS and ESC 0/2 h algorithm combining the concept with bespoke low troponin concentrations. The approach of using absolute troponin deltas over relative troponin delta [RΔ = (AΔ × 100)/TNIH concentration_T=0h], was shown to be superior for early rule out of AMI in multiple studies and is also endorsed by the International Federation of Clinical Chemistry Committee for Clinical Applications of Cardiac Biomarkers (IFCC-CCB), while also advising the evaluation of optimal troponin deltas for individual troponin assays and pathways.^10–13

In light of the recent research into early ACS rule out strategies, we first sought to evaluate the diagnostic accuracy of the Siemens TNIH assay at 0 and 3 h in patients presenting to the ED with suspected cardiac chest pain at the 99th percentile and lower troponin concentrations including establishing the optimal absolute and relative troponin delta for the assay. Our second objective was to externally validate an ‘adapted’ ESC 0/2 h algorithm (applied with testing at 0/3 h), the ESC 0/3 h algorithm, and the High-STEACS pathway with the Siemens TNIH assay.

Methods

Design and setting

This work presents a secondary analysis of data collected in the STAR (sensitive troponin for admission reduction) study, which was nested within the larger Bedside Evaluation of Sensitive Troponin (BEST) study (NIHR portfolio reference UK CRN 18000). The BEST study was a prospective observational study conducted at 19 EDs in the UK. In a phased approach the first site commenced recruitment on 9 February 2015 with the final site completing recruitment on 25 October 2016. For this secondary analysis (STAR study) data from 14 sites were used. An overview of sites included can be found in the Supplementary material online.

Ethical approval was granted by the Research Ethics Committee (reference 14/NW/1344) and all participants provided written informed consent. We have previously published three analysis from the BEST study.^14–16

Study participants

We included adult patients aged ≥18 years presenting to the ED with pain, discomfort or pressure in the chest, epigastrium, neck, jaw, or upper limbs without an obvious non-cardiac source and whom the treating emergency physician felt warranted investigation for suspected ACS.

We excluded patients whose peak symptoms were >12 h before presenting to the ED, patients requiring referral for immediate primary percutaneous coronary intervention due to unequivocal evidence of ST elevation myocardial infarction, patient requiring hospital admissions for another medical condition and patients lacking mental capacity to provide written informed consent.

Data collection and laboratory analysis

All clinical data were prospectively collected by the initial treating ED physician and clinical research nurses at the time of inclusion, using a specifically designed case report form documenting presence and absence of relevant symptoms as well as physical examination findings, 12-lead ECG interpretation, ED diagnosis, patient disposition from the ED, and patient characteristics including their past medical history, current medication use, and cardiovascular risk factors.

Patients in this observation study were treated according to local guidelines for treatment of cardiac chest pain. Local guidelines were checked for their compliance with guidance from NICE and the ESC.

Serum blood samples were routinely taken on arrival to the ED and 3 h later in patients presenting with suspected cardiac chest pain. The blood samples were frozen to −20°C within 4 h and further stored at −70°C within 28 days for analysis of promising biomarkers at a later time. Samples were subsequently analysed for hs-cTnI [Siemens High-Sensitivity Troponin I (TNIH), ADVIA Centaur, 99th percentile upper reference limit overall 46.5 ng/L, female 39.6 ng/L, male 58.0 ng/L; coefficient of variation 10% at 4.50 ng/L]. The performance characteristics of the assay were independently confirmed by the investigators.¹⁷

Pathways

The three pathways that were compared are the ESC 0/3 h pathway, the High-STEACS pathway, and adapted ESC 0/2 h algorithm with their respective AMI rule out criteria are summarized in Figure  1.

ESC 0/3 h pathway

For ruling out myocardial infarction, the ESC 0/3 h pathway proposes cardiac troponin concentrations ≤99th percentile upper reference limit at presentation in patients with symptoms of suspected cardiac chest pain for >6 h; or if the troponin concentration remains ≤99th percentile at 3 h. In accordance with previously published guidance from the ESC Study Group on Biomarkers in Cardiology, rule out was possible if the 3 h troponin is >99th percentile without a significant absolute troponin delta, defined as >50% of the 99th percentile upper reference limit (TNIH: female >20 ng/L and male >29 ng/L). In case of a 0 h troponin concentration >99th percentile the Study Group advocated for a >20% change from the TNIH concentration on arrival. Additionally, patients with a troponin concentration of five times the upper reference limit of the troponin assay (TNIH assay: female 198 ng/L and male 290 ng/L) are considered a rule in regardless of delta change.^3,4 Patients not meeting rule in criteria according to the ESC 0/3 pathway were hence inferred to have ruled out for AMI.

High-STEACS pathway

The High-STEACS pathway was derived and validated with the Abbott Architect STAT hs-cTnI assay and subsequently validated for the Siemens Atellica TNIH assay without recalibration of the thresholds.^9,18 The validated High-STEACS pathway rules out AMI in patients if cardiac troponin levels are <5 ng/L at presentation (unless presenting <2 h after symptom onset) or if patients had a cardiac troponin concentration ≤99th percentile at 3 h and have a 3 h troponin delta of <3 ng/L.⁹

Adapted ESC 0/2 h algorithm

The ESC 2020 0/2 h algorithm using the Siemens TNIH assay was derived in a randomly selected sample from the APACE study and subsequently internally validated in samples of the remaining subjects. The algorithm suggests ruling out AMI in patients with cardiac troponin concentrations <3 ng/L if chest pain symptoms started >3 h prior to ED presentation or a troponin concentration at presentation of <8 ng/L and a 0–2 h troponin delta of <7 ng/L in patients with chest pain for <3 h prior to ED presentation.^7,8 We adapted the algorithm using 0 and 3 h troponin samples with the exact same cut-offs proposed in the 0/2 h algorithm.

Outcomes

In this diagnostic accuracy study, the target condition was AMI. Allocation of the diagnosis of AMI was performed by two independent investigators in accordance with the third universal definition of AMI, based on access to relevant clinical information including ED and subsequent admission medical records, blood results, ECG, and cardiac investigations and procedures within 30 days but blinded to the results of research investigations.¹⁹ If discrepancies could not be resolved by consensus a third independent investigator would be involved.

As a prognostic outcome we also evaluated the incidence of 30-day major adverse cardiac events (MACE), which included index visit and incident AMI and cardiac death within 30 days (in accordance with previous similar studies).

Follow-up

Patients were followed up at 30 days by research nurses checking the inpatient course and contacting the patient via telephone, email, letter, or in person to establish data on the composite outcome of MACE with adjudication of AMI as described and their mortality status including the cause of death if applicable. If repeated attempts of contacting participants directly had failed, follow-up data were obtained from the patients’ primary care practitioners when possible.

Statistical analysis

Baseline characteristics were summarized using mean (standard deviation) or median (interquartile range) as appropriate. For evaluating the diagnostic accuracy of the Siemens TNIH assay at the 99th percentile upper reference limit at 0 and 3 h we calculated sensitivity, specificity, positive predictive value (PPV), and NPV using sex-specific cut-offs (female 40 ng/L and male 58 ng/L). Troponin concentrations were expressed in ng/L and rounded to integers. Both approaches were in accordance with the latest Laboratory Medicine Practice Guidelines (LMPG) on cardiac markers drafted in collaboration with the IFCC TF-CB.²⁰ Paired comparison of diagnostic accuracy measures was performed with McNemar’s test. The overall diagnostic accuracy was summarized by calculating the area under the receiver operator curve (ROC) and graphically displayed. The area under the curve (AUC) of two ROC were compared using the DeLong method.²¹ An absolute troponin delta was mathematically defined as AΔ = |‘3 h-TNIH concentration’ − ‘0 h-TNIH concentration’| and a relative troponin delta as a percentage by the calculating RΔ = (AΔ × 100)/(‘0 h-TNIH concentration’). For calculation of the relative troponin delta in cases with a 0 h troponin concentration of 0 ng/L, the baseline troponin was conservatively adjusted to 0.49 ng/L. We further evaluated the diagnostic accuracy of absolute and relative troponin deltas by calculating sensitivity, specificity, PPV, and NPV and the number of patients ruled out or in at optimal sensitivity and specificity thresholds based on the respective ROC curve analysis. The optimal thresholds were chosen to maximize sensitivity at ≥99% or best possible and to maximize specificity at ≥95%, ≥97.5%, and ≥99% normally corresponding with a PPV >80%. An empirical cut-off [c] was calculated using Youden’s index [J] = maximum {Sensitivity_c + Specificity_c − 1} with a maximum J of 1 representing a perfect test.²² The proportion of patients suitable for AMI rule-out and the proportion with 30-day MACE were summarized using descriptive statistics as appropriate. Statistical analysis was undertaken using SPSS version 23.0 (SPSS Inc., Chicago, IL, USA) and STATA 13.1 (StataCorp LP, College Station, TX, USA) using the diagti, cutpt, roctab, and roccomp commands. Results are reported in accordance with the STARD criteria for diagnostic accuracy studies.²³

Results

Of a total of 1613 patients included in the BEST study, there were 869 consenting participants who had 0 and 3 h samples analysed with the Siemens TNIH assay. We further excluded 57 participants for missing data on reference standard adjudication and 10 for lacking a delta time between ED arrival and the troponin at 0 h (t = 0) blood sample being drawn, resulting in a total of 802 participants included in this secondary analysis (Figure  2).

The baseline characteristics of participants are summarized in Table 1. A total of 111 participants (13.8%) were diagnosed with AMI during their index visit. A total of 114 participants (14.2%) had one or more MACE within 30 days including 60 (7.5%) incident AMI and 4 (0.5%) cardiac deaths.

Diagnostic accuracy at 0 and 3 h at the 99th percentile upper reference limit

At the 99th percentile upper reference limit with sex-specific cut-offs, the Siemens TNIH assays sensitivity for ruling out index visit AMI at 0 and 3 h was 72.1% [95% confidence interval (CI) 62.8–80.2] and 88.3% (95% CI 80.8–93.6), respectively. Diagnostic accuracy was not statistically different when using an overall cut-off (Supplementary material online, Table S1). Further diagnostic accuracy characteristics are summarized in Table 2. Of participants having a troponin level below the 99th percentile at 0 h 31 (4.5%) had an index visit AMI and 33 (4.7%) had a 30-day MACE while at 3 h index visit AMIs were lower at 13 (1.9%) and 30-day MACE was 15 (2.2%). No participant with a troponin concentration below the 99th percentile had a cardiac death within 30 days. The diagnostic accuracy of the Siemens TNIH assay for index visit AMI at 0 and 3 h equates to an AUC of 0.96 (95% CI 0.94–0.98) and 0.98 (95% CI 0.96–0.99), respectively (Figure  3).

In the subgroup sensitivity analysis of 295 early presenters with chest pain onset of <3 h (n = 295) at first troponin sampling the sensitivity for ruling out index visit AMI at the 99th percentile URL with sex-specific cut-offs was 54.8% (95% CI 38.7–70.2) and 88.1% (95% CI 74.4–96.0) at 0 and 3 h sampling, respectively. Additional diagnostic accuracy statistics including for 30-day MACE are summarized in Table 3. The AUC for index visit AMI at 0 h was 0.96 (95% CI 0.94–0.98) compared to 0.99 (95% CI 0.98–1.00) at 3 h with P = 0.002 (Figure  4).

Diagnostic accuracy and optimal troponin deltas of Siemens TNIH assay at 0/3 h

The diagnostic characteristics for the Siemens TNIH assay with an absolute 0/3 h troponin delta of <3 and <7 for ruling out index visit AMI and 30-day MACE and a sex-specific absolute troponin delta of <50% and <20% from the 99th percentile upper reference limit are summarized in Table 4. The AUC was 0.94 (95% CI 0.90–0.97) and 0.76 (95% CI 0.71–0.81) for absolute and relative 0–3 h troponin deltas, respectively (Figure  5). Empirical cut-off estimation established an optimal absolute troponin delta of AΔ = 11 with a Youden index of 0.82 and a relative delta of RΔ = 100% with a Youden index of 0.45. Optimizing the cut-off for absolute and relative delta troponin to achieve maximum possible sensitivity (≥95.5%) for rule out of index visit AMI, resulted in an absolute delta cut-off of ≥1 and a relative delta cut-off of ≥0.05% with both cut-offs ruling out 255 participants. Conversely, optimizing the cut-off of absolute and relative troponin deltas to achieve ≥99% specificity for rule in of index visit AMI resulted in an absolute delta cut-off of ≥38 and a relative delta cut-off of 302.2% achieving a PPV of 93.0% (95% CI 85.4–97.4) and 87.5% (95% CI 73.2–95.8), respectively. Additional optimized cut-offs with responding diagnostic characteristics are summarized in Table 5.

External validation of rule out pathways

In the external validation of the ESC 0/3 h pathway, High-STEACS pathway, and the adapted ESC 0/2 h pathway we found sensitivities of 81.1% (95% CI 80.8–93.6) to 93.7% (95% CI 87.4–97.4) and 98.2% (95% CI 93.6–99.8) for the before mentioned pathways, respectively, when used with the Siemens TNIH assay for ruling out index visit AMI. The results of the external validation, the respective rule out and missed adverse events proportions of the respective three pathways are summarized in Table 6. Additional analysis of all three pathways in early presenters who had their first troponin sampling <3 h after symptom onset is provided in the Supplementary material online, Table S2.

Discussion

In this analysis, we have evaluated the diagnostic accuracy of the Siemens ADVIA Centaur TNIH assay when measured on arrival in the ED and 3 h later for the rule out of AMI and 30-day MACE. We established the assays performance at the 99th percentile, when using absolute and relative 0–3 h troponin deltas including optimized cut-offs. Furthermore, we reported the diagnostic performance of the assay when utilized in the ESC 0/3 h pathway, the High-STEACS pathway, and an adapted ESC 0/2 h algorithm.

Our findings demonstrate that using the Siemens TNIH assay to rule out AMI with tests drawn 3 h apart in the ED using the 99th percentile cut-off has suboptimal sensitivity for clinical purposes. Importantly, the low sensitivity of 72.1% at 0 h and 88.3% at 3 h is in line with earlier reported findings.⁸ However, sensitivity is substantially greater when using lower hs-cTn concentrations set at or just above the limit of quantification of the assay.¹⁷ Furthermore, sensitivity of the 3 h algorithms is enhanced by incorporating the combination of a 0 h cut-off set below the 99th percentile and an absolute delta change, as the ESC 0/2 h and High-STEACS algorithms both do.

With an AUC of 0.94 and 0.76 for absolute and relative 0–3 h troponin deltas, respectively, we have provided further evidence supporting the superior diagnostic accuracy of absolute troponin deltas.^10,24 Furthermore, the AUC for absolute deltas reported by Reichlin et al.²⁵ of the Siemens TNIH at 0.94 is identical to our findings. The sensitivity levels the Siemens TNIH assay reached in our analysis at cut-offs proposed in the subsequently evaluated rule out pathways did not get higher than 87.4% with an absolute 0–3 h delta of <3 ng/L for rule out of index visit AMI with diagnostic performance being even worse when using the other proposed cut-offs or 30-day MACE as outcome. Optimizing cut-offs a maximum sensitivity of ≥95.5% was achievable which would make such a strategy clinically unacceptable, as only 4 out of 10 emergency physicians were found to accept a 1% miss rate of 30-day MACE.²⁶

While the before mentioned strategies on their own do not provide sufficient diagnostic accuracy, pathways like the ESC 0/3 h pathway, High-STEACS pathway, and adapted ESC 0/2 h algorithm evaluated in our analysis provided better performance.

Of the three strategies, the adapted ESC 0/2 h algorithm outperformed the other two pathways with superior sensitivity of 98.2% for AMI and 97.4% for 30-day MACE with corresponding NPVs of 99.6% and 99.4% thus supporting previously published findings on this approach.^7,8 It is worth noting that while we used the originally proposed cut-offs of the 0–2 h algorithm, our analysis used an absolute 0–3 h troponin delta instead of the 0–2 h absolute troponin delta as we had no 2 h blood samples available for analysis. While this in theory could have increased the diagnostic accuracy to some degree we feel it mainly provides reassurance that the adapted ESC 0/2 h algorithm performs equally well at a 0–3 h absolute delta. Further evaluation with 0–2 h samples would be beneficial to confirm the 0–2 h absolute delta.

The High-STEACS pathway showed reasonable sensitivity at 93.7% and a good 98.8% NPV but a marked decrease in sensitivity compared to the 98.0% reported in the original validation of the pathway with the Siemens TNIH assay. While Chapman et al.¹⁸ reported a much lower 90.8% sensitivity and 98.0% NPV when using the ESC 0/3 h pathway, our analysis showed even lower diagnostic accuracy with 79.3% sensitivity and 96.7% NPV. A notable difference potentially affecting diagnostic accuracy between both analyses was the use of the Atellica IM analyser compared to the ADVIA Centaur analyser in our study.

In a related study comparing diagnostic accuracy of the ESC 0/3 h pathway and High-STEACS pathway in conjunction with the Abbott Architect Stat hs-cTnI assay the High-STEACS pathway equally outperformed the ESC 0/3 h pathway with a sensitivity of 97.7% and 89.3% and NPV of 99.5% and 97.9%, respectively.⁹

We further acknowledge that for a definitive rule out of AMI in the ESC 0/3 h pathway, patients would have to meet additional requirements of being pain free, having a GRACE score of <140 and having other differential diagnoses excluded.³ Our data did not allow for assessing those requirements and this may have negatively affected the diagnostic performance of the ESC 0/3 h pathway. While Chapman et al.^9,18 did show an increase in diagnostic accuracy when incorporating those additional requirements in the ESC 0/3 h pathway, diagnostic performance was still inferior to the High-STEACS pathway. Therefore, the additional requirements might partially explain the lower diagnostic accuracy characteristics in our analysis compared to those reported by Chapman et al., but are unlikely to increase the overall ESC 0/3 h pathway performance to a level comparable to the High-STEACS pathway or adapted ESC 0/2 h algorithm.

Using at the time of the study valid third universal definition of myocardial infarction for adjudication rather than the presently used fourth universal definition could present a further limitation.²⁷ Readjudication was not considered feasible considering the resources this would have required. With one major difference being the introduction of the concept of myocardial injury this holds the theoretical potential of wrongly adjudication of AMI in cases of myocardial injury under the old definition. However, the likelihood of overadjudication of AMI under the third universal should be minimal as the inclusion criteria of the study meant that patients had to have chest pain of suggested cardiac nature which would mean rightful adjudication of AMI in cases with a significant rise or fall of troponin.

Conclusion

Using the Siemens TNIH assay, AMI should not be ruled out purely based on the absence of a hs-cTn concentration above the 99th percentile on serial sampling over 3 h, or purely based on the absence of a relative or absolute 0–3 h troponin delta. Using a validated pathway that incorporates both bespoke ‘rule-out’ cut-offs and delta criteria, accuracy improves. In our external validation of three existing pathways the adapted ESC 0/2 h algorithm outperformed the High-STEACS and ESC 0/3 h pathway.

Supplementary material

Supplementary material is available at European Heart Journal – Acute Cardiovascular Care online.

Data availability statement

The data underlying this article cannot be shared publicly due to the privacy of individuals that participated in the study. The data will be shared on reasonable request to the corresponding author if possible.

Funding

This study was sponsored by Manchester University NHS Foundation Trust. Reagents for the purpose of this research were donated by Siemens Healthineers. The Bedside Evaluation of Sensitive Troponin (BEST) study also received funding from the following sources (for separate analyses, not presented here): European Union Horizon-2020 scheme; the Royal College of Emergency Medicine; and Abbott Point of Care.

Conflict of interest: R.B. declares receiving speaker fees from Roche and LumiraDx. The Chief Investigator’s institution has received research funding from Roche and Abbott Point of Care. All other authors declared no conflict of interest.

References