Relative Effectiveness of BNT162b2, mRNA-1273, and Ad26.COV2.S Vaccines and Homologous Boosting in Preventing COVID-19 in Adults in the US

Abstract

Background

Few head-to-head comparisons have been performed on the real-world effectiveness of coronavirus disease 2019 (COVID-19) booster vaccines. We evaluated the relative effectiveness (rVE) of a primary series of mRNA-1273 vs BNT162b2 and Ad26.COV2.S and a homologous mRNA booster against any medically attended, outpatient, and hospitalized COVID-19.

Methods

A data set linking primary care electronic medical records with medical claims data was used for this retrospective cohort study of US patients age ≥18 years vaccinated with a primary series between February and October 2021 (Part 1) and a homologous mRNA booster between October 2021 and January 2022 (Part 2). Adjusted hazard ratios (HRs) were derived from 1:1 matching adjusted across potential covariates. rVE was (1 – HR_adjusted) × 100. Additional analysis was performed across regions and age groups.

Results

Following adjustment, Part 1 rVE for mRNA-1273 vs BNT162b2 was 23% (95% CI, 22%–25%), 23% (95% CI, 22%–25%), and 19% (95% CI, 14%–24%), while the rVE for mRNA-1273 vs Ad26.COV2.S was 50% (95% CI, 48%–51%), 50% (95% CI, 48%–52%), and 57% (95% CI, 53%–61%) against any medically attended, outpatient, and hospitalized COVID-19, respectively. The adjusted rVE in Part 2 for mRNA-1273 vs BNT162b2 was 14% (95% CI, 10%–18%), 13% (95% CI, 8%–17%), and 19% (95% CI, 1%–34%) against any medically attended, outpatient, and hospitalized COVID-19, respectively. rVE against medically attended COVID-19 was higher in adults age ≥65 years (35%; 95% CI, 24%–47%) than in those age 18–64 years (13%; 95% CI, 9%–17%) after the booster.

Conclusions

In this study, mRNA-1273 was more effective than BNT162b2 or Ad26.COV2.S following a primary series during the Delta-dominant period and more effective than BNT162b2 as a booster during the Omicron-dominant period.

During the initial wave of coronavirus disease 2019 (COVID-19) vaccinations in late 2020 and early 2021, 3 severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) vaccines were made available in the United States under Emergency Use Authorizations (EUAs): 2 mRNA vaccines (mRNA-1273 and BNT162b2, available as 2-dose primary series and boosters) and an adenovirus-based vaccine (Ad26.COV2.S; available as a 1-dose primary series and booster). Data from clinical trials indicated high vaccine efficacy through 6 months postvaccination, particularly for mRNA vaccines [1–7]. Following the large-scale rollout of these vaccines to the US population, COVID-19-related hospitalizations and deaths decreased rapidly in the first half of 2021; however, the emergence of the Delta variant followed by the Omicron variant and subvariants resulted in a resurgence of cases [8]. Viral mutations rendering these variants more fit to evade the immune response and increased transmission, combined with host factors associated with greater risk for COVID-19-related morbidity and mortality, contributed to the observed waning of vaccine effectiveness [9–12]. However, administration of an mRNA vaccine booster dose was shown to increase effectiveness against symptomatic and severe disease during both the periods of Delta and Omicron predominance [13]. From October 2021 onwards, mRNA booster doses have been recommended for at-risk populations and subsequently expanded to all individuals over 12 years of age who have completed a primary COVID-19 vaccine series [14, 15]. Immunization with a primary series followed by boosting with mRNA-1273 and BNT162b2 in adults age ≥18 years allows for a direct comparison of vaccine effectiveness. Such data will be important to support vaccination decision-making, particularly in settings where vaccine effectiveness is a criterion for selection.

A limited number of studies have specifically compared the effectiveness of the 2 mRNA vaccines during periods when Delta and Omicron variants predominated [16–18]. However, no study to date has evaluated vaccine effectiveness within the same cohort from primary vaccination through to booster spanning both the Delta- and Omicron-dominant periods. While some comparative effectiveness studies have been published, direct head-to-head effectiveness research is needed, particularly within the context of emerging variants and updated formulations. To address this, we conducted a retrospective longitudinal study using a large integrated electronic health record (EHR) [19] data set to assess the relative vaccine effectiveness (rVE) of mRNA-1273 following primary and booster vaccination vs BNT162b2 and Ad26.COV2.S in preventing COVID-19-related medical encounters (outpatient and hospitalized COVID-19) during periods when Delta and Omicron variants predominated in the United States.

METHODS

Study Design

This retrospective observational longitudinal study was conducted between February 2021 and January 2022 using de-identified (see the Supplementary Data for details) electronic medical records from primary care and specialist clinics with linked pharmacy and medical claims data. Data were evaluated for adults age ≥18 years with a record of receiving a full primary series of mRNA-1273, BNT162b2, or Ad26.COV2.S between February 1, 2021, and October 18, 2021 (Part 1), and a homologous mRNA booster (mRNA-1273 and BNT162b2 only) between October19, 2021, and January 31, 2022 (Part 2; Figure 1). A cutoff date between the primary series and booster vaccination of October19, 2021, was based on the date on which Centers for Disease Control and Prevention (CDC) recommendations for booster doses came into effect. Only individuals who were included in Part 1 were eligible for inclusion in Part 2 of the study. The study was designed, implemented, and reported in accordance with Good Pharmacoepidemiological Practices, applicable local regulations, and the ethical principles laid down in the Declaration of Helsinki. Study findings are reported in accordance with the Reporting of Studies Conducted Using Observational Routinely Collected Health Data recommendations.

Data Source

This analysis was performed on an integrated real-world EHR data set (Veradigm Health Insights) together with pharmacy and medical claims data (Komodo Health Inc., San Francisco, California, USA). This integrated data set has been used extensively to evaluate vaccine effectiveness [20–22] and has been shown to be representative of the US population containing key variables for conducting RWE research [19]. The Veradigm Health Insights EHR database contains data on health care interactions for >55 million patients in the United States whose providers use the Allscripts Touchworks, Allscripts PRO, and Practice Fusion EHRs, including details of prescriptions and vaccinations for both primary care physicians and specialists. Closed medical claims data were included, that is, adjudicated claims within the period in which the patient was continuously enrolled in an insurance plan. As a noninterventional, retrospective database study using a certified Health Insurance Portability and Accountability Act–compliant deidentified research database, approval by an institutional review board was not necessary.

Study Population

Individuals were eligible for inclusion in Part 1 of the study if they were ≥18 years of age, had received 2 doses of mRNA-1273 or BNT162b2 or 1 dose of Ad26.COV2.S (using CPT codes) (Supplementary Table 1), and had linked EHR and claims data together with EHR activity >365 days before the index date (Supplementary Figure 1). The index date was defined as the date of receipt of the second mRNA vaccine dose or the date of receipt of the single Ad26.COV2.S dose. The cohort entry date was defined as 14 days after the index date. Receipt of a homologous COVID-19 vaccine booster (Supplementary Table 1) between October 19 and January 31, 2022, was an additional criterion for Part 2 of the study; individuals who received a booster dose before October 19, 2021, were excluded. Exclusion criteria included receipt of a heterologous COVID-19 vaccine, no record of the second dose of mRNA-1273 within 28 ± 5 days of the first dose or BNT162b2 within 21 ± 5 days of the first dose, evidence of previous confirmed COVID-19 infection between January 1, 2020, and the beginning of the follow-up period, homologous vaccination between the index and cohort entry dates, and missing birth year or gender.

Fully vaccinated mRNA-1273 recipients, that is, those who had completed the primary series, were matched 1:1 with individuals from the comparator vaccine groups (BNT162b2 and AD26.COV2.S for Part 1; BNT162b2 only for Part 2) based on sex, geographic region, and race. Age was matched within 5-year age groups, and index date was matched ±5 days. mRNA-1273 recipients could be included in both primary series comparisons (BNT162b2 and Ad26.COV2.S).

Exposure and Outcome Ascertainment

EHR data, together with pharmacy and medical claims data, were used to identify individuals vaccinated between the cohort entry and end of intake dates (Supplementary Figure 1). Follow-up for assessment of outcomes of interest was performed from the cohort entry date until the earliest of the following events: first occurrence of an outcome of interest; end of the observation period; or disenrollment from their medical/pharmacy plan. The cohort entry date for Part 2 was the date of receipt of the booster dose, and the follow-up period ranged from the cohort entry date to January 31, 2022.

The primary outcome was any medically attended COVID-19, defined as any outpatient or hospitalized medical encounter with a COVID-19 diagnosis. As secondary outcomes, hospitalized and outpatient COVID-19 medical encounters were evaluated separately (see Supplementary Table 2 for defining codes). Safety was not evaluated in this study.

Statistical Analysis

Covariate balance at baseline for each comparator vaccine vs mRNA-1273 was assessed using standardized mean differences (SMDs) before any adjustments. Kaplan-Meier plots with associated 95% CIs were generated to assess rVE over time. Right censoring was defined as the end of the follow-up period, with a maximum follow-up time of 265 days post–primary series and 90 days postbooster. For each of the comparisons, unadjusted hazard ratios (HRs) were estimated using a univariate Cox regression model with no covariates. Adjusted HRs were calculated using a multivariable Cox regression model, adjusting for covariates with an SMD ≥0.1 (Supplementary Tables 3 and 4). rVE was defined as (1 – adjusted HR) × 100. Statistical analyses were performed using R Statistical Software (version 4.1.3) [23] and the survival (version 3.2–13; Therneau, 2021) and MatchIt (version 4.3.2; Ho et al., 2021) packages.

Additional Exploratory Analyses

Exploratory analysis was also performed for the primary and secondary outcomes of interest by age group (18–64 years, 65–74 years, ≥65 years, and ≥75 years) and region (Midwest, Northeast, South, West, or unknown). A protocol-specified sensitivity analysis was also performed for the outcome of laboratory-confirmed SARS-CoV-2 infection for Parts 1 and 2, defined as a positive lab result as recorded in the EHR.

RESULTS

Of the ∼15.9 million patients included in the linked EHR claims data set who had been fully vaccinated with a primary series BNT162b2, mRNA-1273, or Ad26.COV2.S between February 1 and October 18, 2021, 4 404 091 were included in the first part of this analysis (Supplementary Figure 2a). Among these, 2 092 304 received BNT162b2, 1 788 220 received mRNA-1273, and 523 567 received Ad26.COV2.S. Overall, 1 529 930 individuals who received mRNA-1273 were matched with BNT162b2 recipients, and 484 795 were matched with Ad26.COV2.S recipients (Table 1; Supplementary Table 3).

In total, 4 022 367 individuals received a booster dose of BNT162b2 or mRNA-1273 between October 19, 2021, and January 31, 2022 (Supplementary Figure 2b). Of these, 509 014 BNT162b2 and 430 268 mRNA-1273 recipients were eligible for inclusion. Of these, 368 100 matched individuals who received a homologous booster dose of mRNA.1273 or BNT162b2 were included in the second part of the analysis (Table 2; Supplementary Table 4).

Cumulative Incidence of COVID-19

Analysis of the primary outcome of any medically attended COVID-19 showed a lower cumulative incidence for mRNA-1273 compared with BNT162b2 or Ad26.COV2.S after the primary series, with adjusted HRs of 0.77 (95% CI, 0.75–0.78) and 0.50 (95% CI, 0.49–0.52), respectively (Table 3; Supplementary Figure 3). This corresponded to rVE estimates of 23% (95% CI, 22%–25%) and 50% (95% CI, 48%–51%), respectively.

For both vaccine comparisons, adjusted HRs across the secondary outcomes analyzed reflected those for medically attended COVID-19 (Table 3). For mRNA-1273 vs BNT162b2, adjusted rVE ranged from 23% to 28% for hospitalized and outpatient COVID-19, and for mRNA-1273 vs Ad26.COV2.S, adjusted rVE was 50% for both measures.

Analysis of mRNA-1273 vs BNT162b2 after the booster dose resulted in an rVE of 14% (95% CI, 10%–18%) for the primary outcome of any medically attended COVID-19 (Table 3; Supplementary Figure 4). Estimates of adjusted HR for outpatient and hospitalized COVID-19 also showed a higher VE for mRNA-1273 compared with BNT162b2, with an rVE of 13% (95% CI, 8%–17%) and 19% (95% CI, 1%–34%), respectively.

Analysis by Age Group

Overall, 75.8%–76.8% of individuals were in the 18–64 years age group across comparisons of the primary series, and 87.7% for the booster dose (Supplementary Table 5).

In the exploratory analysis by age, after the primary series, adjusted HRs for mRNA-1273 vs BNT162b2 were highest for the primary and secondary outcomes in the 18–64 years age group (0.78–0.88), although confidence intervals overlapped between all age groups (18–64, 65–74, and ≥75), suggesting no clear age effect (Table 4). Similarly, the data suggested no clear age effect for mRNA-1273 vs Ad26.COV2.S, with adjusted HRs across outcomes and age groups. Across all outcomes and for all age groups, higher point estimates for rVE of mRNA-1273 were observed as compared with the other 2 vaccines (rVE range, 12%–67%).

After the booster dose, a trend for an age effect was observed against any medically attended and outpatient COVID-19, with lower HRs in individuals ≥65 years of age compared with younger adults. Point estimates were also lower for the other outcomes analyzed across age groups; however, confidence intervals overlapped (Table 4).

Analysis by Region

In the exploratory analysis by region, no apparent differences in adjusted HRs were evident when assessed by region for either the primary series or booster dose (Table 5). For mRNA-1273 vs BNT162b2, rVE for the primary outcome of any medically attended COVID-19 after the primary series ranged from 20% to 24% across regions, with overlapping confidence intervals between regions for secondary outcome measures. Results for the mRNA-1273 vs Ad26.COV2.S comparison also appeared similar across regions, with rVE for any medically attended COVID-19 ranging from 46% to 52%. After the homologous booster, rVE for mRNA-1273 vs BNT162b2 ranged from 4% to 29% across the 3 measures, with overlapping confidence intervals between regions.

Sensitivity Analysis: Lab-Confirmed COVID-19

Analysis of lab-confirmed COVID-19 was performed as a sensitivity analysis, as rates of testing decreased substantially during the latter months of the study period. As with the findings against medically attended, outpatient, and hospitalized COVID-19, point estimates for effectiveness were higher for mRNA-1273 than either BNT162b2 or Ad26.CoV.2 (rVE 28% and 52%, respectively) (Supplementary Table 6). Subset analysis suggested no clear differences by age or region. Analysis of the booster dose was confounded by low rates of testing (Supplementary Table 6) but also suggested increased effectiveness of mRNA-1273 vs BNT162b2.

DISCUSSION

To our knowledge, this is the first analysis of the rVE of mRNA-1273 compared with other COVID-19 vaccines that follows the same cohort of individuals through primary and booster vaccination. In order to emulate real-life decisions as much as possible, cohort entry dates into the booster phase of the study were based on recommended CDC dates, and therefore analysis of the primary series was truncated from this point forward. Over a period where Delta predominated, a primary series of mRNA-1273 was more effective than either BNT162b2 or Ad26.COV2.S in preventing any medically attended, outpatient, hospitalized, and lab-confirmed COVID-19. In addition, evaluation of the impact of a homologous booster during the Omicron-dominant period demonstrated greater effectiveness with mRNA-1273 vs BNT162b2 against any medically attended, outpatient, hospitalized, and lab-confirmed COVID-19. These differences appeared consistent by region and age; however, an mRNA-1273 booster appeared to have increased rVE against medically attended COVID-19 compared with BNT162b2 in a subgroup analysis of older adults (≥65 years of age).

The results following the primary series are consistent with findings from previous real-world evaluations of rVE of COVID-19 vaccines. In a study of health records of US veterans who received a primary series of COVID-19 vaccine, mRNA-1273 was more effective than BNT162b2 in adults age <65 years and >65 years against symptomatic infection (57.3% vs 22.5% and 36.2% vs −23.3%, respectively), hospitalization (83.1% vs 57.0% and 64.7% vs 1.7%, respectively), and intensive care unit admission or death (84.4% vs 66.4% and 73.8% vs 29.3%, respectively) [16]. Consistent with these findings, the authors also demonstrated mRNA1273 to be more effective than BNT162b2 in veterans with >1 chronic disease. In addition, evaluation of COVID-19 hospitalizations across 21 US states between March and August 2021 showed significantly higher vaccine effectiveness of a primary series of mRNA-1273 (93%; 95% CI, 91%–95%) compared with BNT162b2 (88%; 95% CI, 85%–91%) or Ad26.COV2.S (71%; 95% CI, 56%–81%) [24].

A number of real-world studies have also demonstrated a clear impact of booster doses on vaccine effectiveness, particularly against the Omicron variant [12, 13, 25–28]. In a comparative study of booster vaccinations based on data from the OpenSAFELY-TPP database in the United Kingdom during the period of Delta and Omicron dominance, HRs for mRNA-1273 vs BNT162b2 were 0.92 (95% CI, 0.91–0.92) and 0.67 (95% CI, 0.58–0.78) for lab-confirmed and hospitalized COVID-19, respectively, 12 weeks after booster vaccination [29]. Similarly, in a study in veterans in the United States, 16-week risk of COVID-19-related outcomes was lower in recipients of a third dose of mRNA-1273 vs BNT162b2, particularly for documented infection (excess events for BNT162b2 vs mRNA-1273 was 45.5; 95% CI, 19.4–84.7; per 10 000 persons) [27].

In our study, exploratory analysis suggested an increased benefit of mRNA-1273 vs BNT162b2 booster vaccination against medically attended COVID-19 in older adults (≥65 years). These differences in effectiveness would have a meaningful impact on preventing and reducing the burden of COVID-19 in individuals at higher risk of more severe disease. Consistent with these findings, an analysis by Mayr et al. [16] demonstrated mRNA-1273 to be more effective than BNT162b2 in older veterans and those with chronic diseases. Evaluation of long-term antibody persistence following primary series and homologous boosting with mRNA COVID-19 vaccines has shown statistically higher antibody titers and persistence for mRNA-1273 compared with BNT162b2 against both the ancestral strain and subsequent variants [30]. The observed relative differences in effectiveness in this study may be, in part, due to differences in their immunogenicity [31]. Regardless of immune measures evaluated, mRNA1273 has been observed to elicit the greatest immune response, followed by BNT162b2, and both mRNA vaccines are significantly more immunogenic than Ad26.COV2.S [30–32]. This trend has also been observed following the bivalent booster, with higher specific IgG and T-cell responses compared with BNT162b2 bivalent booster [32]. While the mechanisms underlying these immunogenicity differences are not fully understood, they could potentially be related to differences in vaccine antigen content, epitope-specific antibody responses, and levels of T-cell response [31, 33, 34]. Although not assessed specifically in older adults, higher immunogenicity and antibody persistence may contribute to the observed greater effectiveness of mRNA-1273, which is particularly important in combination with immunosenescence in the older age group. While there was no significant difference in vaccine effectiveness between the mRNA vaccine boosters in younger adults (18–64 years), point estimates of HRs favored mRNA-1273 and may have been significant with larger sample sizes, as the majority of booster recipients prioritized during the period of this analysis were in the older age group (≥50 years). Future analysis including more recipients from the younger age groups will help to confirm this finding, as well as any differences in vaccine effectiveness within groups at high risk for severe disease.

A key strength of this analysis was the richness of the available data from EHRs, which include demographics, comorbidities, laboratory results, and health care encounters in both outpatient and hospital settings. This allowed close matching of individuals across multiple potential confounding variables. Additionally, the large number of individuals included in the analysis allowed robust subgroup assessments by age group and region. However, it should be noted that some age groups still had small sample sizes or low numbers of cases (eg, lab-confirmed COVID-19 in patients ≥75 years) or lab tests in the booster phase. The results of this analysis should be interpreted within the context of the retrospective nature of the study. In the absence of randomization, there may be unmeasured differences between groups, which may have confounded vaccine effectiveness estimates. An additional limitation of this type of study is that misclassification of exposure and outcomes is potentially more common than in a randomized clinical trial, although misclassification of vaccine administration was unlikely because of comprehensive recording of vaccine administration in our database and the strict time window for administration of the second dose. Furthermore, while clinical cases were determined in this analysis from EHR records, rather than directly identified following a positive polymerase chain reaction (PCR) test, these were confirmed by a record of a positive PCR test in >90% of cases. The vast majority (>90%) of COVID-19 mRNA boosters administered in the United States were the same as administered in the primary series [35]. As such, we only evaluated effectiveness after a homologous booster vaccination; results may potentially differ in individuals who received a heterologous primary series and booster. Finally, there is a potential bias due to right-censoring of the data. As individuals included received vaccinations up until the end date, the follow-up period for some individuals was potentially very short, meaning that some medically attended cases that occurred outside of the follow-up window are not included in this analysis.

In summary, receipt of a primary series of mRNA-1273 vaccine resulted in lower risk of any medically attended, outpatient, hospitalized, and lab-confirmed COVID-19 compared with BNT162b2 or Ad26.CoV2.S during a Delta-dominant period. Boosting with mRNA-1273 also resulted in reduced risk of any medically attended, outpatient, and hospitalized COVID-19 compared with BNT162b2 during the Omicron-dominant period.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Acknowledgments

The authors would like to thank the following individuals for their contribution: Editorial assistance was provided by Dr. Jenny Engelmoer (Sula Communications BV).

Financial support. This work was supported by Moderna.

Data availability. Individual-level data reported in this study are not publicly shared. Upon request, and subject to review, Veradigm may provide the de-identified aggregate-level data that support the findings of this study. De-identified data (including participant data as applicable) may be shared upon approval of an analysis proposal and a signed data access agreement. Individual-level data reported in this study are not shared publicly, but they are shared fully with regulatory agencies.

Patient consent. This retrospective cohort study does not include factors necessitating patient consent.

References

Author notes