Extracorporeal life support in cardiac arrest: a post hoc Bayesian re-analysis of the INCEPTION trial

Abstract

Aims

Previously, we performed the multicentre INCEPTION trial, randomizing patients with refractory out-of-hospital cardiac arrest (OHCA) to extracorporeal cardiopulmonary resuscitation (ECPR) or conventional cardiopulmonary resuscitation (CCPR). Frequentist analysis showed no statistically significant treatment effect for the primary outcome; 30-day survival with a favourable neurologic outcome (cerebral performance category score of 1–2). To facilitate a probabilistic interpretation of the results, we present a Bayesian re-analysis of the INCEPTION trial.

Methods and results

We analysed survival with a favourable neurologic outcome at 30 days and 6 months under a minimally informative prior in the intention-to-treat population. Effect sizes are presented as absolute risk differences (ARDs) and relative risks (RRs), with 95% credible intervals (CrIs). We estimated posterior probabilities at various thresholds, including the minimal clinically important difference (MCID) (5% ARD), based on expert consensus, and performed sensitivity analyses under sceptical and literature-based priors. The mean ARD for 30-day survival with a favourable neurologic outcome was 3.6% (95% CrI −9.5–16.7%), favouring ECPR, with a median RR of 1.22 (95% CrI 0.59–2.51). The posterior probability of an MCID was 42% at 30 days and 42% at 6 months, in favour of ECPR.

Conclusion

Bayesian re-analysis of the INCEPTION trial estimated a 42% probability of an MCID between ECPR and CCPR in refractory OHCA in terms of 30-day survival with a favourable neurologic outcome.

Trial registration

Clinicaltrials.gov (NCT03101787, registered 5 April 2017).

Graphical Abstract

Open in new tabDownload slide

Introduction

Out-of-hospital cardiac arrest (OHCA) has a dismal prognosis. Early (bystander) cardiopulmonary resuscitation (CPR) and defibrillation attempts offer the best chance of survival. However, when early defibrillation attempts fail to restore spontaneous circulation, the chance of survival declines rapidly.¹ Extracorporeal CPR (ECPR) can restore circulation in patients with refractory cardiac arrest, potentially improving survival and neurologic outcomes.² Still, randomized trials evaluating the efficacy and effectiveness of ECPR in refractory OHCA have yielded varying results.^3–5

As the number of patients eligible for ECPR is low, expected differences—rather than clinically important differences—have been used in the sample size calculations of these studies to ensure their feasibility.^6–8 The use of expected differences to calculate the sample size may result in trials that are underpowered to detect a clinically relevant treatment effect. A probabilistic analysis using Bayes’ theorem facilitates a more nuanced interpretation of the results of such trials by estimating the probability of a given effect size.

Recently, we published the Early Initiation of Extracorporeal Life Support in Refractory OHCA trial (INCEPTION).⁵ This was a multicentre, pragmatic trial assessing the effectiveness of ECPR in refractory OHCA. The sample size was calculated based on an expected survival difference of 22%.⁷ Frequentist statistical analysis found no statistically significant difference between ECPR and conventional CPR (CCPR) with respect to the primary outcome; 30-day survival with a favourable neurologic outcome [defined by a cerebral performance category score of 1 or 2 (CPC 1 or 2)].⁵

Here, we present a Bayesian re-analysis of the INCEPTION trial to enable a probabilistic interpretation of the results. In particular, we assessed the posterior probability of a minimal clinically important difference (MCID) between ECPR and CCPR, which was determined at 5% absolute risk difference (ARD), based on published expert consensus.⁹

Methods

Ethics statement

The protocol, design, and informed consent procedure have been comprehensively reported previously,⁷ and was registered at Clinicaltrials.gov (NCT03101787, registered 5 April 2017). The research protocol was approved by the leading centre’s institutional review board [Maastricht University Medical Center+ (NL58067.068.16/METC162039)]. In resuscitation research, study participants often remain incapacitated for a prolonged time, meaning proxy consent is accepted as the standard procedure. As such, the concept of deferred consent was applied, which means written informed consent was obtained from all patients or patients’ legal surrogates after inclusion in the study.¹⁰ Members of the study team performed these informed consent conversations. In all cases, where subjects regained consciousness at any point in time, confirmation of the proxy consent was sought from the subject self. This study adhered to the Reporting of Bayes used in Clinical Studies (ROBUST) criteria.¹¹

The INCEPTION trial

The INCEPTION trial was an investigator-initiated, randomized controlled trial (RCT) comparing ECPR with CCPR for patients with refractory OHCA. The trial was conducted in 10 Dutch cardiosurgical centres and had a pragmatic design,¹² which was described elsewhere.⁵ For the INCEPTION trial, adult patients with a witnessed arrest and primary ventricular tachycardia of fibrillation were included until the age of 70 years. Refractory OHCA was defined as no return of spontaneous circulation (ROSC) after 15 min, despite bystander CPR and one or more defibrillating shocks by an automated external defibrillator or emergency medical services. If randomized to ECPR, ECPR was implanted upon arrival in the emergency department, provided there was no ROSC, and no post-randomization contraindications had emerged. The primary outcome was 30-day survival in a favourable neurologic outcome, defined as CPC 1 or 2. Among others, 6-month survival with a favourable neurologic outcome CPC 1–2 was a secondary outcome.⁷

Sample size and power calculations were based on an expected survival with a favourable neurologic outcome of 8% in the CCPR group and an estimated survival with a favourable neurologic outcome of 30% in the ECPR group⁷ [22% ARD, relative risk (RR) 3.75].

Bayesian inference

The Bayesian framework is increasingly applied to enhance the interpretation of time- and resource-consuming clinical trials, which are at risk of underpowered sample sizes.¹³ As such, clinically relevant treatment effects may not surface, although they may be present. In general, the principal components of a Bayesian analysis are the incorporation of prior knowledge (the prior) into new findings (the likelihood) in order to formulate the posterior probability of a treatment effect (the posterior). The unbiased formulation of the prior is of utmost importance to guarantee transparency and reproducibility, while maintaining objectivity. As such, multiple priors are generally formulated, which range from a ‘sceptical’ attitude towards a treatment to an ‘enthusiastic’ view. The incorporation of these priors is aimed to assess the robustness of the observed results, in the light of different conceptions of a treatment, which can also differ between treating physicians. Furthermore, the least biased prior is the minimally informative prior, which has only a negligible effect on the posterior (which is, therefore, determined by the likelihood).

Then, the Bayesian framework also facilitates the evaluation of a clinically relevant treatment effect, because the posterior probability of any treatment effect threshold can be estimated. This is in contrast to the frequentist framework, in which the presence of any effect is tested, regardless of its clinical relevance. However, the threshold for clinical relevance may also be a matter of subjectivity and should always be specifically formulated and ideally be supported by evidence or expert consensus. For a more detailed elaboration on the merits of the Bayesian clinical trial analysis, we refer to previous literature.^13–15 A detailed description of the applied methodology in the current trial is outlined in the following sections.

Outcomes of the current analysis

We estimated the posterior probabilities of 30-day survival with a favourable neurologic outcome (CPC 1–2) in the intention-to-treat population of the INCEPTION trial. In addition, we assessed survival with a favourable neurologic outcome at 6 months (maximum follow-up), as a secondary outcome. Based on a published expert consensus statement,⁹ the threshold for the MCID in patients with shockable OHCA was defined as a 5% ARD in survival with a favourable neurologic outcome. However, as the MCID may still be perceived subjectively and was not necessarily based on patients undergoing ECPR, various other ARD thresholds were studied as well for the purposes of sensitivity. These thresholds include a 7.5, 10, and 15% ARD.

Prior specification

Prior distributions were reported using the natural logarithmic function (log) RR and its log RR standard deviation (SD) with corresponding 95% CrIs. We used a minimally informative prior for the primary analysis. As such, the posterior probability distribution derived from this prior was primarily informed by the results from the INCEPTION trial.

Sensitivity analyses were performed using a sceptical prior, and evidence-based informative priors, which are also presented in Table 1:

• The sceptical prior was derived from the expected survival rate in the CCPR group that was used in the sample size calculation of the INCEPTION trial (8%).⁵ A prior probability that assumed that there was no difference between both groups and a 10% probability of the MCID (ARD ≥5%) was considered to represent a sceptical prior view towards the effectiveness of ECPR for OHCA. This is in agreement with the recently proposed characteristics of a sceptical prior as advocated by de Grooth and Elbers.¹⁶

• The first informative prior for the primary outcome was based on the 30-day survival rate in a recent meta-analysis by Scquizzato et al.¹⁷ This meta-analysis incorporated results from randomized trials and matched cohort studies, including ‘real-world registries’. These data can be considered as a comprehensive summary of the pragmatic evidence of ECPR for OHCA, which is in line with the pragmatic design of the INCEPTION trial.

• The second informative prior for the primary outcome was based on the 30-day survival with a favourable neurologic outcome rate of the Prague-OHCA study.⁴ Although this study was a rather explanatory trial, and as such not fully comparable with the INCEPTION trial, it resembled INCEPTION’s design, particularly with regard to pre-hospital randomization.

Posterior probability reporting, probability calculation, and statistical software

To enhance the intuitive interpretation of the results of the current analysis, the log RRs and their SDs were converted to RR and ARD, accompanied by the 95% CrI and visually presented in plots. The probabilities of any benefit (RR > 1.0, ARD > 0%), any harm (RR < 1.0, ARD < 0%), and a MCID (ARD > 5%) were presented under the assumption of the various pre-specified priors for the primary outcome and the secondary outcome. For all probability calculations, the survival rate in the CCPR group of INCEPTION was used as a reference point.⁵

All Bayesian analyses were performed using JASP (JASP Team 2022, version 0.16.3 for MacOS), an R-based statistical programme with Bayesian capabilities, applying the ‘abtest’ package,¹⁸ incorporating Markov chain Monte Carlo sampling (3 chains, 500 burn-ins, 10 000 saved iterations per chain).

Results

Baseline characteristics of the study population and the results of the INCEPTION trial are published elsewhere extensively.⁵ Briefly, 160 patients were randomized, of whom 134 were included in the final intention-to-treat analysis (CCPR n = 64, ECPR n = 70). The 30-day survival rate with CPC 1–2 was 20% in the ECPR group (n = 14/70 patients) and 16% in the CCPR group [n = 10/62 available patients, odds ratio (OR) 1.4, 95% confidence interval (CI) 0.5–3.5, P = 0.518]. The survival rate with CPC 1–2 at 6 months was 20% in the ECPR group (n = 14/70 patients) and 16% in the CCPR group (n = 10/63 available patients, OR 1.3, 95% CI 0.5–3.3).⁵

Thirty-day neurologically favourable survival in Bayesian re-analysis

Under a minimally informative prior, the estimated mean ARD was 3.6% (95% CrI −9.5–16.7%) in favour of ECPR, with an estimated median RR of 1.22 (95% CrI 0.59–2.51). The posterior probability of a clinically important difference was 41.7%. The posterior probability of any benefit of ECPR, regardless of the clinical relevance of the effect size (defined as RR > 1.0, ARD > 0%), was 70.5%. Consequently, the probability of any harm (RR < 1.0) was 29.5% (Table 2 and Figure 1). Under a minimally informative prior, the posterior probability of an ARD exceeding 7.5, 10, and 15% for 30-day survival with favourable neurology was 27.8, 16.9, and 4.4%, respectively (see Supplementary material online, S1). The full posterior probability distribution under a minimally informative prior is presented in Figure 2.

A sensitivity analysis under a sceptical prior resulted in a median ARD of 1.8% (95% CrI −7.5–11.1%) with an estimated median RR of 1.10 (95% CrI 0.68–1.80). The posterior probability of a MCID was estimated at 25.0% and the probability of any harm (RR < 1.0) was 35.2%.

Finally, the posterior probability of a minimal clinically relevant difference was 20.8% when informed by data from a recent meta-analysis.¹⁷ This posterior probability was estimated at 82.0%, when informed by the results from the Prague-OHCA study (Table 3).⁴ Of note, the posterior probability of any benefit (RR > 1.0) exceeded 94% under both informed priors. All probability distributions for the primary outcome are shown in Figure 3.

Six-month neurologically favourable survival

Under the minimally informative prior, the median RR of survival with ECPR was 1.23 (95% CrI 0.60–2.49), and the mean ARD was 3.7% (95% CrI −9.1–16.4%; Table 2). Consequently, the probability of an MCID was 42.1%. The probability for any benefit, regardless of its clinical relevance, was 71.6% and the probability of any harm was 28.4% (Table 3).

In a sensitivity analysis under the sceptical prior, a 24.8% posterior probability of the MCID was estimated. Based on literature-based priors, these posterior probabilities for the MCID ranged between 63.7 and 90.8% (Table 3). All probability distributions for the secondary outcome are shown in Figure 4. The posterior probability of various other ARD thresholds is presented in Supplementary material online, S1.

Discussion

The INCEPTION trial was a multicentre, pragmatic RCT performed in 10 Dutch cardiosurgical centres that evaluated the effectiveness of extracorporeal CPR in refractory OHCA.^5,7 Using the commonly applied frequentist approach, no statistically significant difference was found between ECPR and CCPR, in the original analysis. Numerically, the absolute risk reduction in the ECPR group approximated the MCID that was established by expert consensus.⁹ As the study was powered to detect an expected survival effect of 22%,⁷ the trial may have been underpowered to detect a clinically relevant survival effect. To estimate the probability of an MCID, we performed a Bayesian re-analysis, which found a posterior probability of a clinically important benefit of ECPR of 41.7%.

Traditionally, randomized trials are analysed using frequentist statistics, limited as they are by the dichotomous conclusion that a trial is either positive or negative. However, when the sample size of a study is too small to detect an MCID, a potentially clinically important observation may not surface. In contrast, the Bayesian framework allows the estimation of the posterior probability of any effect size, facilitating an intuitive interpretation of the results of clinical trials that establish an apparent—yet non-statistically significant—treatment effect.¹⁴ It has been pointed out that Bayesian analyses may be of particular interest in critical care research, because many trials are not sufficiently powered to address clinically relevant differences in the primary outcome.^13,15

The incorporation of prior probabilities in Bayesian inference reflects the fact that new data update evidence or clinical experience already existing prior to a study. However, when one aims to determine the posterior probability almost exclusively based on the original trial’s own results (the likelihood), a minimally informative prior can be applied. As such a prior has only a negligible effect on the posterior probability, the posterior probability of a Bayesian analysis under a minimally informative prior closely reflects the actual trial results. We considered the posterior probability estimations under the minimally informative prior as the most relevant outcomes for the current Bayesian re-analysis, as there are no studies that methodologically resemble the pragmatism of the INCEPTION trial.

Apart from the minimally informative prior, we incorporated sceptical and informative priors in several sensitivity analyses. These sceptical priors assess the robustness of study results by incorporating a realistic degree of scepticism, preferably derived from existing evidence, but their specification is arbitrary. We derived our sceptic belief from a recent expert statement, stating that a sceptical prior should be compatible with a mere 10% probability of the MCID, while assuming no difference.¹⁶ Under this sceptical prior, the probability of a clinically important effect decreased from 41.7 to 25.0% for 30-day survival with a favourable neurologic outcome.

Informative priors can be derived from published literature. The choice of informative priors may be controversial, as inappropriate priors might substantially and unjustly affect posterior probability.¹⁶ It is important to derive the priors from data that are relevant and comparable with the data of the study that is analysed. Prior to the INCEPTION trial, two RCTs addressed the efficacy of ECPR in OHCA; the Advanced R² Eperfusion STrategies for Refractory Cardiac Arrest (ARREST) trial and the Prague-OHCA study.^3,4 Both studies sparked considerable enthusiasm for ECPR in OHCA.¹⁹ As the Prague OHCA study was most similar to the INCEPTION trial, particularly by virtue of pre-hospital randomization, we derived the first informative prior from the Prague-OHCA study. Furthermore, to resemble the multicentre real-world design of the INCEPTION trial in an informative prior, we derived data from a recent meta-analysis.¹⁷ Of note, this meta-analysis contained data from the Prague-OHCA study and the ARREST trial but also incorporated ‘real-world’ (propensity-matched) registries.

The posterior probabilities estimated from the trial data and the informative priors consistently resulted in a high probability of any ECPR benefit (which is not necessarily clinically important). The incorporation of informative priors yielded widely divergent posterior probabilities of a clinically important difference, which seems to be primarily dictated by the prior distribution. Indeed, the >90% posterior probability of a clinically important treatment effect (resulting from the posterior distribution based on the analysis under the informative prior) seems to be dominated by the incorporated prior.^6,17 To fully comprehend the interaction between prior distributions and posterior probabilities in this re-analysis, it is important to review the prior and likelihood distributions for each separate analysis (such as included in the Figures 3 and 4 of this Bayesian re-analysis). Still, the high likelihood of a clinically important treatment effect under such an enthusiastic prior resembles the belief of many treating physicians and is fuelled by positive efficacy trials.^3,4

The INCEPTION trial was a pragmatic multicentre trial, aimed to evaluate the effectiveness of ECPR, rather than its efficacy.¹² Case load varied markedly between centre and ECPR protocols were aligned with institutional routines and protocols, which also varied between centres.^5,7,19 In contrast, ARREST and Prague-OHCA were explanatory single-centre trials,¹² incorporating highly dedicated ECPR teams and a considerable case load. These trials have established the potential clinical efficacy of ECPR for OHCA.

However, the INCEPTION trial shows that these excellent ECPR results are not easily reproduced. The Bayesian analysis under a non-informative prior, which solely incorporates data from the INCEPTION trial, shows a 17–42% probability of a 5–10% increase in survival with a favourable neurological outcome, even in a well-organized healthcare system in one of Europe’s richest and most developed countries. This may give rise to the question if ECPR should even still be performed. Healthcare systems that consistently report clinical effectiveness of ECPR have specifically focused on ECPR and invested many material and human resources in optimizing the time to extracorporeal membrane oxygenation. However, such investments may not be feasible for all, and their proportionality should be weighed against case load and achieved incremental effectiveness. Hitherto, the minimum requirements to establish or maintain a clinically effective ECPR programme remain unknown. Several modelling studies have claimed that ECPR for OHCA is highly cost-effective,^20,21 but most of these studies assume a very optimistic incremental effectiveness and do not take into account the background costs of running a 24/7 ECPR-ready system.

Nevertheless, in centres that are willing and able to tackle the limitations of the INCEPTION trial—such as case load, exposure, and the prolonged time between hospital arrival and start of cannulation—these drawbacks may not apply, and ECPR may still be feasible. Therefore, we foresee that other multicentre RCTs will be conducted to study the effectiveness of ECPR in refractory OHCA in the near future. To ensure feasibility, a Bayesian analysis of the primary endpoint may be of particular interest in these studies, for which the results of the INCEPTION trial may serve as an informative prior.

The current re-analysis focused on an alternative statistical approach to put the findings of the original INCEPTION trial publication into context,⁵ without dichotomizing the trial into a positive or negative study. Consequently, we evaluated the posterior probability of various effect size thresholds and have attempted to define the clinical relevance of such a treatment effect. Still, the clinical relevance of a treatment effect is subjective and may differ based on geographical location, culture, institutional practice, and even between physicians. We used an MCID that was derived from an international survey among experts who contended that 5% absolute risk reduction was relevant in patients with OHCA secondary to ventricular fibrillation. However, the response rate in this survey was only 50%, and determination of an MCID may also be affected by the impact of a treatment on patients and the healthcare system. In the light of this, it may be argued that 5% is a somewhat liberal threshold for a clinically important difference. Therefore, we have also specifically reported the posterior probability of clinical benefit at thresholds of 7.5, 10, and 15% ARD (see Supplementary material online, S1).

Limitations

The most important limitation of this study is that this post hoc analysis was not part of the initial statistical analysis plan and the priors were not defined beforehand. Indeed, Bayesian re-analyses are at risk of confirmation bias.²² We were aware of this risk and incorporated a reasonably sceptical prior, in addition to the literature-based prior. This sceptical prior represented a sceptical attitude towards the effectiveness of ECPR for OHCA. Furthermore, the assessment of any difference could also be interpreted as confirmation bias. Therefore, we specifically assessed the consensus-based MCID as the primary outcome.

Conclusions

The current Bayesian re-analysis demonstrated that the posterior probability of a clinically important ECPR benefit in terms of 30-day survival with a favourable neurological outcome for refractory OHCA was 42% in the INCEPTION trial. Although the use of Bayesian inference is not yet widespread in the medical literature, we believe that the probabilistic analysis of trial data mirrors clinical reasoning and should be applied more widely to enhance the interpretation of results generated by resourceful clinical trials.

Supplementary material

Supplementary material is available at European Heart Journal: Acute Cardiovascular Care.

Author contributions

S.H.: conceptualization, formal analysis, investigation, methodology, writing original draft, reviewing, and editing. A.F.v.d.K.: investigation, methodology, writing original draft, reviewing, and editing. A.G.: formal analysis, methodology, validation, supervision, reviewing, and editing. J.F.H.U.: formal analysis, investigation, methodology, reviewing, and editing. I.C.C.v.d.H.: methodology, validation, supervision, reviewing, and editing. T.S.R.D. and M.M.S.: conceptualization, data curation, formal analysis, validation, reviewing, and editing. J.G.M. and R.L.: conceptualization, methodology, validation, resources, supervision, reviewing, and editing. M.C.G.v.d.P.: conceptualization, formal analysis, funding acquisition, investigation, methodology, resources, supervision, validation, writing original draft, reviewing, and editing.

Funding

Getinge (commercial) and ZonMw (governmental) provided financial support to conduct the original trial (M.C.G.v.d.P.). These providers did not have any influence on the study design, data collection, data analysis, interpretation of the data, drafting of the manuscript, or decision to submit the original trial. Furthermore, they had no influence on the decision to perform and submit this secondary analysis.

Ethical approval

The research protocol was approved by the leading centre’s institutional review board [Maastricht University Medical Center+ (NL58067.068.16/METC162039)].

Data availability

In the light of open science, all data and settings in the statistical software package JASP are made available through https://github.com/samuelheuts/INCEPTION_Bayes.

References

Author notes