Effectiveness of Messenger RNA–based Vaccines During the Emergence of the Severe Acute Respiratory Syndrome Coronavirus 2 Omicron Variant Free

Abstract

Messenger RNA (mRNA)–based vaccines have demonstrated significant protection against coronavirus disease 2019 (COVID-19) compared with no vaccine in clinical as well as observational studies [1–4]. The spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Delta variant, resurgence of COVID-19 infections, and concern about waning antibody levels among vaccinated persons led the US Food and Drug Administration (FDA) to authorize a third dose of BNT162b2 (Pfizer-BioNTech) or mRNA-1273 (Moderna) 6 months after completing the primary series for vaccinated persons at high risk of severe disease or exposure to COVID-19 [5, 6]. Soon after, several states expanded eligibility of booster doses to all adults as part of a continued effort to control COVID-19 [7–9]. On 19 November 2021, the FDA expanded eligibility for booster vaccine doses to all individuals aged ≥18 years after completion of the primary vaccine series [10]; eligibility was adjusted to everyone aged ≥12 years for recipients of BNT162b2 [11]. By 1 April 2022, the US Centers for Disease Control and Prevention (CDC) estimated that more than 65% of the US population was fully vaccinated and that 45% of fully vaccinated persons had received an additional dose of vaccine [12].

The emergence and rapid dissemination of the SARS-CoV-2 Omicron variant in December 2021 raised new questions about the effectiveness of mRNA-based vaccines against this novel strain. Early reports from South Africa and the US CDC suggested protection against the Omicron variant, though these studies were limited to hospitalized individuals [13, 14]. Additional studies in frontline healthcare workers and for self-reported symptomatic COVID-19 following the emergence of the Omicron variant have indicated continued effectiveness of mRNA-based vaccines [15, 16]. The effectiveness of mRNA-based vaccines across the clinical spectrum of COVID-19 severity as well as the effectiveness of completing the primary series of 2 doses in contrast to receiving a third dose have yet to be assessed. Observational studies in Israel demonstrated the effectiveness of a third dose of BNT162b2 in preventing post-vaccination COVID-19 compared with completion of only the primary series [17, 18]. However, these studies predated the emergence of the Omicron variant; vaccine effectiveness of mRNA-based vaccines in the era of Omicron variant predominance has yet to be fully evaluated in the United States.

We leveraged electronic health records from the Veterans Health Administration (VHA) to estimate the effectiveness of BNT162b2 and mRNA-1273 vaccines in preventing post-vaccination COVID-19 infection following the emergence of the Omicron variant.

METHODS

The VHA is the largest integrated health system in the United States, providing healthcare services at 1293 facilities [19]. Individual-level clinical records are parsed and imported into the VHA Corporate Data Warehouse, which is used to conduct observational studies as well as to monitor multilevel operations. The VHA implemented vaccination for COVID-19 beginning in December 2020. The study period was 1 December 2021 to 12 March 2022. For study inclusion, an individual needed to have had at least 1 primary care visit at a VHA facility during calendar year 2020 and not have had a documented positive SARS-CoV-2 polymerase chain reaction (PCR) test before 1 December 2021. Individuals who received a dose of Ad26.COV2.S, who received doses of both BNT162b2 and mRNA-1273, or who were admitted to long-term care facilities were excluded. Three subgroups were considered: persons with no documented administration of an mRNA-based vaccine (unvaccinated persons), individuals who had received 2 doses of BNT162b2 or mRNA-1273 before 1 December 2021 (primary-series recipients), and persons who received an additional dose of BNT162b2 or mRNA-1273 after completing the primary series at least 14 days before 1 December 2021 (third-dose recipients).

For each vaccine type, third-dose recipients were matched with equal numbers of primary-series recipients and unvaccinated persons on multiple demographic and clinical covariates including age, sex, race/ethnicity, comorbidities (summarized by the Elixhauser comorbidity score [20, 21]), and US county of residence. Third-dose recipients and primary-series recipients were matched on an additional covariate that corresponded to the calendar week in which the second dose of vaccine was received. To account for healthcare-seeking behavior, subgroups were also matched on number of SARS-CoV-2 PCR tests received before the study start date.

Three outcomes were considered: documented SARS-CoV-2 infection (defined as a positive SARS-CoV-2 PCR test), COVID-19 hospitalization (defined as documented SARS-CoV-2 infection within 21 days before admission to an inpatient unit in an acute care VHA facility), and COVID-19 death (defined as death within 30 days after documented SARS-CoV-2 infection). For each outcome and vaccine, individuals were followed from recruitment until the earliest date of outcome, death, or end of the study period. Follow-up of primary series recipients was also halted if a third vaccine dose was received during follow-up.

The Kaplan-Meier estimator was used to calculate the 100-day cumulative incidence (risk) of outcomes for each subgroup and vaccine. Vaccine effectiveness (1 – risk ratio) was calculated for 3 measures: primary-series compared with unvaccinated, third-dose compared with unvaccinated, and third-dose compared with primary-series. Nonparametric bootstrapping with 1000 samples was used to calculate 95% confidence intervals (CIs). All analyses were performed in R version 4.10 [22].

The Stanford University Institutional Review Board (Public Health Surveillance in the Department of Veterans Affairs) approved this project, and written informed consent was waived.

RESULTS

At the study start, 1 237 990 individuals who received BNT162b2 and 1 404 183 persons vaccinated with mRNA-1273 met eligibility criteria. Among these, 366 293 (29.6%) and 309 050 (22.0%) individuals received a third dose of BNT162b2 or mRNA-1273, respectively. Additionally, by 1 December 2021, 1 821 245 individuals were unvaccinated. After matching, the BNT162b2 group consisted of 221 267 third-dose recipients and the mRNA-1273 group included 187 507 third-dose recipients linked to equal numbers of unvaccinated and primary-series recipients who had not received a third dose. Baseline demographic characteristics were similar across matched populations within vaccine groups (Table 1). Participants were older persons, mostly male, and non-Hispanic White. Most (>80%) participants did not undergo SARS-CoV-2 PCR testing before the start of follow-up. About one quarter of study participants in each group had an Elixhauser comorbidity score of 10 or greater. Among vaccinated participants, more than 50% had completed the primary series at least 250 days before the study start date. The median time that had elapsed from receipt of the third dose before start of follow-up was 48 days for the BNT162b2 group and 28 days for the mRNA-1273 group.

In both vaccine groups, documented SARS-CoV-2, COVID-19 hospitalization, and COVID-19 death occurred more frequently in unvaccinated persons compared with those who completed the primary series or received a third dose (Table 1, Figures 1–3). In the analysis of BNT162b2, the 100-day cumulative incidence of documented SARS-CoV-2 infection was 2.44% (95% CI, 2.37–2.50) in those unvaccinated, 1.82% (95% CI, 1.76–1.89) in primary-series recipients, and 1.27% (95% CI, 1.23–1.32) in third-dose recipients. The 100-day cumulative incidence of COVID-19 hospitalization in the BNT162b2 group was 0.67% (95% CI, .63–.70) in those unvaccinated, 0.31% (95% CI, .29–.34) in primary-series recipients, and 0.12% (95% CI, .11–.14) in third-dose recipients. The 100-day risk of COVID-19 death in the BNT162b2 group was 0.13% (95% CI, .11–.14) in those unvaccinated, 0.061% (95% CI, .050–.073) in primary-series recipients, and 0.013% (95% CI, .008–.018) in third-dose recipients.

Figure 1.

Cumulative incidence of documented SARS-CoV-2 infections by vaccination status and manufacturer. Shaded areas describe 95% confidence intervals. Abbreviation: SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Open in new tabDownload slide

Figure 2.

Cumulative incidence of COVID-19 hospitalizations by vaccination status and manufacturer. Shaded areas describe 95% confidence intervals. Abbreviation: COVID-19, coronavirus disease 2019.

Open in new tabDownload slide

Figure 3.

Cumulative incidence of COVID-19 deaths by vaccination status and manufacturer. Shaded areas describe 95% confidence intervals. Abbreviation: COVID-19, coronavirus disease 2019.

Open in new tabDownload slide

The findings in the mRNA-1273 analysis were similar: 100-day cumulative incidence of documented SARS-CoV-2 infection was 1.99% (95% CI, 1.93–2.06) in those unvaccinated, 1.21% (95% CI, 1.15–1.26) in primary-series recipients, and 0.76% (95% CI, .72–.80) in third-dose recipients. Cumulative incidence of COVID-19 hospitalization was 0.49% (95% CI, .45–.52) in those unvaccinated, 0.16% (95% CI, .14–.18) in primary-series recipients, and 0.059% (95% CI, .048–.070) in third-dose recipients. The 100-day risk of COVID-19 death was 0.093% (95% CI, .079–.107) in those unvaccinated, 0.032% (95% CI, .023–.041) in primary-series recipients, and 0.008% (95% CI, .004–.012) in third-dose recipients.

Compared with no vaccination, estimated effectiveness of BNT162b2 against documented SARS-CoV-2 infection was 47.8% (95% CI, 45.2–50.3) for a third dose and 25.3% (95% CI, 21.8–28.7) for the primary series; against COVID-19 hospitalization, it was 81.8% (95% CI, 79.2–84.2) for a third dose and 52.9% (95% CI, 47.8–57.6) for the primary series; and against COVID-19 death, it was 89.6% (95% CI, 85.0–93.6) for a third dose and 50.7% (95% CI, 37.9–61.6) for the primary series (Table 2). A third dose of BNT162b2 compared with the primary series was 30.1% (95% CI, 26.2–33.7), 61.4% (95% CI, 55.0–67.1), and 78.8% (95% CI, 67.9–87.5) effective against documented SARS-CoV-2 infection, COVID-19 hospitalization, and COVID-19 death, respectively.

Estimated effectiveness of a third dose of mRNA-1273 compared with no vaccination was 61.9% (95% CI, 59.4–64.4) for documented SARS-CoV-2 infection, 87.9% (95% CI, 85.3–90.2) for COVID-19 hospitalization, and 91.4% (95% CI, 86.4–95.6) for COVID-19 death (Table 2). Effectiveness of the primary series compared with no vaccination was 39.5% (95% CI, 35.8–43.0) for documented SARS-CoV-2 infection, 66.7% (95% CI, 61.4–71.6) for COVID-19 hospitalization, and 65.6% (95% CI, 52.8–76.3) for COVID-19 death. Compared with the primary series, a third dose of mRNA-1273 was 37.1% (95% CI, 32.2–41.7), 63.5% (95% CI, 53.7–71.6), and 75.0% (95% CI, 55.4–88.0) effective against documented SARS-CoV2-infection, COVID-19 hospitalization, and COVID-19 death, respectively.

DISCUSSION

National surveillance of SARS-CoV-2 variants indicates that the Omicron variant has exceeded more than 90% of the weekly proportion of variants since January 2022 and more than 99% since February 2022, heralding the era of Omicron variant predominance [23]. We estimated the effectiveness of BNT162b2 and mRNA-1273 vaccines following the emergence and spread of the SARS-CoV-2 Omicron variant in the largest healthcare system in the United States and demonstrated substantial protective effect of mRNA-based vaccines against severe post-vaccination COVID-19 infections during this new era. Compared with no vaccination, a third dose was more than 80% effective against COVID-19 hospitalization and death. Protective effects were also observed for documented SARS-CoV-2 infection, though with lower estimates than for COVID-19 hospitalization or death. A third dose of vaccine demonstrated higher effectiveness against both outcomes compared with the primary series. Our findings of effectiveness of mRNA-based vaccine against COVID-19 during the Omicron surge are comparable to recent reports in South Africa [24] and the United States [13, 15, 16]. Our study expands on these reports by estimating effectiveness across the clinical spectrum of post-vaccination COVID-19 infections for both BNT162b2 and mRNA-1273 vaccines as well as demonstrating added protection of a third dose compared with the primary series.

We observed that all measured COVID-19 infection outcomes occurred less frequently in persons who received the third dose of vaccine compared with individuals who completed the primary series. These observations are consistent with differences in predicted pathways of vaccine-mediated immunity against the Omicron variant, this is, heightened antibody evasion facilitates mild illness, but an unimpaired cellular immune response maintains protection against severe infection [14, 25–27]. Reduced frequency of outcomes in persons who received a third dose compared with the primary series might be explained by restored antibody levels that would otherwise have been reduced after completing the primary series. However, our observations might be explained by differences in exposures. Those sufficiently concerned about post-vaccination infection to seek an additional vaccine dose might also choose less risky behaviors than nonrecipients; therefore, differences in general preventive behaviors between third-dose recipients and nonrecipients might confound the frequency of observed outcomes. Similarly, older persons or those with a higher number of comorbidities might not engage in as many social activities compared with younger, healthier individuals and, consequently, might not experience comparable transmission risk. Our study also suggests longitudinal protection of the primary series of mRNA-based vaccines against severe COVID-19 infection, with COVID-19 hospitalization occurring in <0.4% and COVID-19 deaths occurring in <0.1% of primary-series recipients of BNT162b2 or mRNA-1273 in 100 days of observation following emergence of the Omicron variant.

Though we observed considerable protection of mRNA-based vaccines against COVID-19 infection, it is unclear whether this improvement affects broader efforts to reduce COVID-19 transmission. Early studies in the United Kingdom demonstrated that household contacts of vaccinated cases had lower risk of symptomatic secondary infection compared with household contacts of unvaccinated cases [28]. However, after the emergence of the Delta variant, the protective effect on household contacts was less clear. A large observational study in the United Kingdom demonstrated that vaccinated persons carry peak viral loads that are similar to those of unvaccinated persons but for a shorter duration. Additionally, the secondary attack rate among household contacts did not differ by vaccination status of the index case, and transmission was observed to occur among fully vaccinated index case–contact pairs [29]. The heightened evasion to neutralizing antibodies coupled with evidence of rapid spread globally suggest that transmission of the Omicron variant might not be substantially affected by previous immunity or vaccination. At the population level, it is likely that a third dose of an mRNA-based vaccine will remain important in preventing severe infection, particularly among high-risk individuals. The effect on curtailing transmission is less certain and warrants further study.

Our findings are subject to several limitations. Clinical records for patients who received care in facilities external to the VHA might not be available in VHA databases unless these services were ordered by VHA providers and paid for by the VHA; therefore, these testing episodes and outcomes would be missed in our analysis. Although the VHA issued national testing guidelines, differences across VHA facilities in testing assays and local policies or approaches to testing may contribute to variability in detection of vaccine breakthrough events; some events might have been missed or misclassified. Our study population consisted of predominantly older (median age, 75 years), male persons (98%) receiving care at VHA facilities; therefore, our results might not be generalizable to the larger US population. Positive PCR tests in hospitalized persons and decedents may be incidental findings not associated with severe COVID-19 infection; misclassification might affect the accuracy of our estimates for these outcomes. Though we sought to control for health-seeking behavior as well as demographic and clinical covariates associated with COVID-19, unmeasured confounders might affect our findings. To reduce residual confounding, we excluded long-term care residents; our findings might not be applicable to this subgroup. To focus on recent users of VHA services, we selected eligible persons as those who received a primary care visit in 2020; vaccine effectiveness among irregular users of VHA services and those who enrolled after 2021 might differ compared with our estimates. We excluded persons with a documented positive SARS-CoV-2 PCR test before December 2021. Our estimates of vaccine effectiveness thereby exclude the effects of infection-induced immunity, though it is possible that some enrolled individuals might have had previously undiagnosed or undocumented COVID-19 infection. Though the VHA performs passive surveillance of SARS-CoV-2 variants, confirmation of the causative variant for each outcome was not possible; however, both internal VHA genomic surveillance data (unpublished data) as well as variant proportions reported by the CDC confirm the predominance of the Omicron variant during the study period [23]. We did not generate estimates during periods of predominance of other variants, though unobserved, temporally associated variables would adversely affect the accuracy of comparative effectiveness across variant-specific periods. Given the observational nature of this study, data describing additional biomarkers, timing of exposures, symptoms, and the specific variants occurring in vaccine breakthrough events were unavailable; therefore, we were unable to assess the importance of these factors with post-vaccination infection.

In summary, we observed substantial effectiveness of the BNT162b2 and mRNA-1273 vaccines against measured COVID-19 infection outcomes following the emergence of the Omicron variant. Compared with unvaccinated individuals, those who received a third dose or who completed the primary series experienced fewer episodes of documented SARS-CoV-2 infection, COVID-19 hospitalization, and COVID-19 death. A third dose was more effective than the primary series for most outcomes. Clinicians and public health administrators should consider these findings in the broader context of patient- and population-level efforts to combat COVID-19 in this new chapter of the pandemic.

Notes

Author contributions. Conceptualization: A. S., G. O., and M. H.; data curation: A. S.; formal analysis: A. S.; methodology: A. S.; project administration: A. S., G. O., and M. H.; resources: A. S., G. O., and M. H.; software: A. S.; supervision: M. H.; visualization: A. S.; writing (original draft): A. S.; and writing (review and editing): A. S., G. O., and M. H.

Data sharing. Due to US Department of Veterans Affairs (VA) regulations, the analytic datasets used for this study are not permitted to leave the VA firewall without a data use agreement. This limitation is consistent with other studies based on VA data.

Disclaimer. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the VA or the US government.

References

Author notes