How important are placebo controls in clinical trials of interventional procedures?

This editorial refers to ‘Randomized trials of invasive cardiovascular interventions that include a placebo control: a systematic review and meta-analysis’^†, by L. Lauder et al., on page 2556.

The dictionary definition of the placebo effect is "improvement in the condition of a patient that occurs in response to a treatment [the placebo] but cannot be considered due to the specific treatment used".¹ The earliest application of a placebo dates to the 16th century and has been attributed to the Catholic Church which applied a control intervention to assess the efficacy of exorcism.² The placebo effect began receiving serious medical attention with the advent of randomized clinical trials (RCTs), and in 1955 Beecher published a landmark article in which he described a beneficial response to placebo in nearly 35% of patients.³ Although initially believed to be a solely psychological reaction, it is now appreciated that the placebo effect is a complex psychobiological phenomenon attributable to the overall therapeutic context.² In regard to clinical medicine, the placebo effect is one of the most prevalent and powerful confounding variables in clinical trials.

It has long been considered requisite to include a control group in RCTs of pharmacological agents. However, it has been more challenging, and therefore much less common, to include a placebo control group in trials of interventional procedures. Safety has, of course, been the major concern in applying an invasive intervention of no possible benefit to the patient. Accordingly, many interventional procedures have been introduced into medical, and especially cardiovascular, practice based upon data from clinical trials lacking a true placebo control.

Recently a number of RCTs of invasive therapies have been performed that have included a true placebo control group. Two trials, in particular, have had a major impact on the management of the disorders that they were developed to treat, and upon the necessity of true controls. Based upon studies without an invasive placebo procedure, it has long been ‘gospel’ that percutaneous catheter intervention (PCI) was an effective treatment for angina. The ORBITA Trial⁴ was meticulously carried out to test this hypothesis by performing a diagnostic catheterization as the control for PCI while blinding both the patient and the trial evaluators. The unexpected result was a lack of a significant difference in the primary endpoint of exertional capacity between the two arms. Similarly, numerous studies without invasive controls had demonstrated the efficacy of renal artery sympathetic denervation to reduce elevated blood pressure. However, Simplicity 3 tested this procedure in a trial with control patients that underwent catheterization, and found no difference between treated and placebo patients.⁵ Clearly, both studies had significant limitations that coloured the interpretation of the findings. Nevertheless, these studies, in aggregate, created a crack in the bedrock foundation of the validity of RCTs of interventional procedures lacking a placebo control.

It is in this setting that Lauder et al.⁶ undertook to examine both the frequency and degree of efficacy of RCTs of interventional procedures that included a true invasive control, as published in this issue of the European Heart Journal. Although a perfect test of the precise influence of an interventional placebo upon an RCT would involve performing the exact protocol with and without the control, that will require future study. Alternatively, one could compare the results of RCTs testing a similar intervention that did and did not have a placebo control. Rather the investigators conducted a retrospective systematic review and meta-analysis of RCTs of cardiovascular interventions in which a randomized interventional placebo group was included. They analysed the odds ratio for studies with dichotomous primary outcomes and assessed the magnitude of effect by means of calculating a standardized mean difference (SMD) between arms for studies with continuous outcomes. Although never specifically stated, it appears that the goal was to test the hypothesis that RCTs with a placebo frequently yield results indicating that the intervention is ineffective or minimally effective.

To a certain extent, the results were anticipated. Given the documented impact of the placebo effect, it is not surprising that interventional studies in which the placebo effect is eliminated will be more challenged to show efficacy. Indeed, Lauder et al. found that only 10 of 30 trials reported a statistically significant benefit of the intervention in regard to the primary endpoint. Of the 16 trials with a continuous endpoint, such as blood pressure, SMD analysis revealed only a small or moderate magnitude of benefit for the intervention in 7 of the trials. The authors conclude simply that the additional treatment effect of invasive cardiovascular interventions compared with placebo controls was small in most trials.

The obvious reason why placebo control groups are not included routinely in RCTs of interventional cardiovascular therapies is the concern for patient safety. Investigators are loath to subject patients to potentially injurious procedures with no chance of potential benefits. Even when the investigators are disposed to such risk, Institutional Review Boards are even more conservative in relation to avoiding patient harm. In this regard, the results of this meta-analysis provide relatively little information. Only 26 of 30 studies had pre-defined safety endpoints that were reported. While six of these studies reported no increase in safety issues in the placebo group, nine studies found a higher incidence of adverse events in the placebo than in the treated group. No further information was provided. Moreover, the authors indicated that most of the studies included in their analysis examined procedures that were judged to be relatively low risk. Clearly, the risk of harm will continue to be a major impediment to including a placebo control in RCTs of interventional procedures.

As is true of nearly all studies, the report by Lauder et al. has limitations. A confounding variable comparable with the placebo effect in RCTs is the conscious or subconscious bias of the investigators. The authors acknowledge that they could not fully assess the blinding of investigators of the studies, and that evaluating the risk of bias was problematic. However, one would intuitively assume that any such bias would favour a beneficial effect of the intervention. Some trials may have ultimately been underpowered to find a salutary effect, and the absence of a cohort that did not undergo any intervention prevented the determination of the placebo effect in the studies analysed. The SMD analysis carried out was blunt, with only three categories of efficacy at 0.20 for mild, 0.50 for moderate, and 0.80 for large levels. The authors indicate that their data suggest that the actual effect of applied interventional procedures supported by RCTs without placebo controls may be less than assumed. While this may well be true, it is speculative, and no direct comparative data are provided. In fact, two trials of cardiac resynchronization therapy (CRT), one with a placebo device (MIRACLE)⁷ and one without (CARE-HF),⁸ had rather similar results, with a Minnesota Quality of Life score of ∼10. In this regard, the reason a placebo control was included in some interventional procedure trials and not others may represent a confounding variable. It is likely that the interventional therapies for which a placebo was included in the RCT represented procedures for which the equipoise was greatest and the risk of adverse events was least. If this were true, it might be expected that a substantial number of such studies would yield negative or neutral findings, or results that were marginal in magnitude. As such, the data from the present study could be difficult to relate to RCTs of interventional therapies that did not include a placebo control.

Over the past two decades randomized, blinded clinical trials have been the source of evidence-based medicine and the cornerstone of guideline-directed medical therapy. Nowhere in medicine has this been more true than for cardiovascular disease. However, RCTs of interventional procedures have often lacked a true invasive placebo control, and were therefore susceptible to confounding by the placebo effect. The potential impact of this effect is particularly true with patient-reported endpoints, and has been dramatically demonstrated in two recent trials in which uncontrolled data were not confirmed when a placebo was employed.^{4  ,  5} The current report by Lauder et al. sheds additional light on this issue, and finds that when a placebo control was included in an RCT, the benefit of the intervention was often absent or modest. Their data suggest that, while it is possible to safely employ invasive controls with the appropriate care, adverse effects frequently occur in the placebo group and represent a continuing concern. Given the recent experience, reports of RCTs of interventional techniques are often greeted with the question of whether or not a placebo control was included. Those trials in which it was not may receive a more reserved acceptance, especially when patient-reported or influenced endpoints are involved. A consensus has emerged that, whenever possible, every effort should be made to include a true placebo control in RCTs of interventional procedures.

Conflict of interest: none declared.

The opinions expressed in this article are not necessarily those of the Editors of the European Heart Journal or of the European Society of Cardiology.

Footnotes

References