Effectiveness of the 2023–2024 Omicron XBB.1.5-containing mRNA COVID-19 Vaccine (mRNA-1273.815) in Preventing COVID-19–related Hospitalizations and Medical Encounters Among Adults in the United States

Abstract

Background

This study aimed to evaluate the vaccine effectiveness (VE) of mRNA-1273.815, a 2023–2024 Omicron XBB.1.5-containing mRNA COVID-19 vaccine, at preventing COVID-19–related hospitalizations and any medically attended COVID-19 in adults.

Methods

In a linked electronic health record–claims dataset, we identified US adults (≥18 years) who received the mRNA-1273.815 vaccine (exposed cohort) between 12 September and 15 December 2023, matched 1:1 to individuals who did not receive a 2023–2024 updated COVID-19 vaccine (unexposed cohort). Cohorts were balanced using inverse probability of treatment weighting on demographics, vaccination and infection history, and underlying medical conditions. Study cohorts were followed until 31 December 2023 for COVID-19–related hospitalizations and medically attended COVID-19. Cox regression was used to estimate hazard ratios and VE. Subgroup analyses were performed for adults ≥50 years, adults ≥65 years, and individuals with underlying medical conditions.

Results

Overall, 859 335 matched pairs of mRNA-1273.815 recipients and unexposed adults were identified. The mean (standard deviation) age was 63 (16) years. More than 60% of individuals in both cohorts had an underlying medical condition. Among the overall adult population, VE was 60.2% (95% confidence interval, 53.4–66.0) against COVID-19–related hospitalization and 33.1% (30.2–35.9) against medically attended COVID-19 over a median follow-up of 63 (interquartile range: 44–78) days. VE estimates by age and underlying medical conditions were similar.

Conclusions

These results demonstrate the significant protection provided by mRNA-1273.815 against COVID-19–related hospitalizations and any medically attended COVID-19 in adults, regardless of vaccination history, and support Centers for Disease Control and Prevention recommendations to stay up-to-date with COVID-19 vaccination to prevent COVID-19–related outcomes, including hospitalizations.

COVID-19 continues to be the most common cause of severe outcomes due to viral respiratory illness in the United States and was associated with >900 000 hospitalizations and 75 000 deaths in 2023 [1–3]. In 2023, the average weekly incidence of COVID-19–associated hospitalizations and deaths surpassed the average weekly incidence of influenza and respiratory syncytial virus, according to the US Centers for Disease Control and Prevention (CDC) [1]. The burden of COVID-19 is not limited to the acute phase because even patients with mild disease can develop conditions, including long COVID, that last months following the initial infection [4–6].

Due to the emergence of novel SARS-CoV-2 viral variants and the observed impact on vaccine performance, the US Food and Drug Administration approved and authorized updated formulations of COVID-19 vaccines that were designed to target the circulating Omicron XBB.1.5 variant in the fall of 2023 [7]. Early data suggest that Omicron XBB.1.5-containing vaccines elicit robust and diverse neutralizing antibody responses against XBB.1.5 and other recent circulating variants, including JN.1 [8, 9]. Several studies have examined the vaccine effectiveness (VE) of the 2023–2024 Omicron XBB.1.5-containing mRNA COVID-19 vaccines [10–14]; however, data on the real-world effectiveness of the mRNA-1273.815 (mRNA-1273 XBB.1.5 containing vaccine) vaccine specifically are limited [15]. This study aimed to evaluate the VE of the mRNA-1273.815 vaccine against COVID-19–related hospitalizations and any medically attended COVID-19 in adults aged 18 years or older.

METHODS

Data Sources

This study leveraged electronic health record (EHR) data from the Veradigm Network EHR linked to healthcare claims sourced from Komodo Health spanning 1 March 2020 through 31 December 2023. The EHR data are sourced from ambulatory/outpatient primary care and specialty settings, and the claims data include inpatient, outpatient, and pharmacy sources. This integrated dataset has been previously characterized and used previously in COVID-19 epidemiology and VE research [2, 16–18]. The dataset used in this study contains only deidentified data as per the deidentification standard defined in Section §164.514(a) of the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Privacy Rule. As a noninterventional, retrospective database study using data from a certified HIPAA-compliant deidentified research database, approval by an institutional review board was not required.

Study Design Overview

We conducted a retrospective, noninterventional cohort study of mRNA-1273.815 VE in adults who were aged ≥18 years and residing in the United States. Adults vaccinated with mRNA-1273.815 during the vaccine intake period were identified and matched in a 1:1 ratio to adults with no evidence of receiving a 2023–2024 Omicron XBB.1.5-containing COVID-19 vaccine. Baseline patient characteristics (covariates) were measured, and inverse probability of treatment weighting (IPTW) was used to further reduce potential confounding by creating a weighted study sample based on the predefined covariates (see subsection Covariates). Individuals in the vaccinated (exposed) and unvaccinated (unexposed) cohorts were followed for the 2 key outcomes, COVID-19–related hospitalizations and any medically attended COVID-19, during the follow-up period. Cox regression models were used to calculate hazard ratios (HR) and 95% confidence intervals (95% CI), which were converted to VE estimates. The study design is shown in Figure 1.

Study Population

Individuals with a record of vaccination with mRNA-1273.815 (50 mcg) during the vaccine intake period (12 September 2023–15 December 2023) were eligible for inclusion in the exposed cohort. Individuals without a record of vaccination with any 2023–2024 updated COVID-19 vaccine during the vaccine intake period were eligible for inclusion in the unexposed cohort.

The index date of the individuals in the exposed cohort was the date of their first mRNA-1273.815 vaccination. Individuals in the unexposed cohort were assigned multiple potential index dates based on the index dates of all possible direct matches in the exposed cohort using the following matching criteria: age on 12 September 2023, sex, race, ethnicity, region, week of last claims or EHR activity, and receipt of a bivalent BA.4/BA.5 COVID-19 vaccine between 1 September 2022 and 11 September 2023.

We required that individuals have continuous enrollment in medical and pharmacy claims from 12 September 2022, through 7 days after the index date (cohort entry date [CED]). Individuals with any of the following were excluded: (1) missing sex or region data, (2) evidence of COVID-19 diagnosis or treatment between 90 days before and 7 days after the index date, or (3) evidence of vaccination with any COVID-19 vaccine in the 60 days before the index day through the CED (excluding the index vaccination in the exposed group). Age was a required field in the underlying data file. Individuals were required to have ≥1 day of follow-up after the CED. Follow-up began the day after the CED and continued until the earliest of the following: outcome of interest, the end of continuous enrollment, receipt of a subsequent COVID-19 vaccine, or end of the follow-up period (31 December 2023).

After these selection criteria were applied, the final matches were created as follows. First, an individual in the exposed cohort was randomly selected. Then, if they had exactly 1 match in the unexposed cohort who met all the selection criteria, they were assigned that individual as a match, and both were removed from further matching. If they had multiple potential matches who met all the selection criteria, they were randomly assigned 1 match, and both were removed from further matching. If the exposed individual had zero remaining matches who met all the selection criteria, they were excluded from the study. This process was repeated until all individuals in the exposed cohort were either matched to someone in the unexposed cohort or excluded from the study. Individuals in the unexposed cohort who were not matched to an individual in the exposed cohort were excluded. The final index date for individuals in the unexposed cohort was the same as their matched mRNA-1273.815 counterpart. By matching on the week of latest activity and then assigning an index date based on their final match, we ensured that follow-up time was similar between the 2 cohorts.

The following subsets of the overall study population were defined for exploratory analyses: adults aged ≥50 years, adults aged ≥65 years, and adults aged ≥18 years with at least 1 underlying medical condition that increased the risk of severe outcomes from COVID-19 as defined by the CDC [17]. The codes used to identify 2023–2024 Omicron XBB.1.5-containing COVID-19 vaccines, any COVID-19 vaccination, and COVID-19 outcomes are reported in Supplementary Tables 1–3.

Outcome Measures

The primary outcome of interest was COVID-19–related hospitalization, and the secondary outcome of interest was any medically attended COVID-19. Each outcome was evaluated independently so that an individual could be counted as having both a COVID-19–related hospitalization and medically attended COVID-19.

COVID-19–related hospitalizations were identified from hospitalization claims with documentation of COVID-19 diagnosis in any position. Any medically attended COVID-19 included various clinical settings such as inpatient admissions, outpatient visits (emergency department visits, urgent care visits, office visits, and telemedicine visits), and laboratory results. These were identified from an EHR or medical claim with either a COVID-19 diagnosis in any setting or a positive laboratory test result in the ambulatory care setting. This may include positive test results for asymptomatic infections. The codes used to identify COVID-19 diagnosis are reported in Supplementary Table 3.

Covariates

The study used covariate information including demographics (age on 12 September 2023, sex [female, male], race [Black, White, other, and unknown], ethnicity [Hispanic, non-Hispanic, and unknown]), insurance type (commercial, Medicaid, Medicare Advantage, Medicare Fee-For-Service, other, and unknown), region (Midwest, Northeast, South, West, and unknown), and month of index. Time since last COVID-19 vaccination and time since last COVID-19 infection were captured using a look-back period that started on 1 March 2020. The number of outpatient visits and the number of hospitalizations were captured using a 12-month preperiod.

The presence of underlying medical conditions associated with an increased risk of severe COVID-19 outcomes, as defined by the US CDC, was captured [19]. These included asthma, cancer, cerebrovascular disease, chronic kidney disease, chronic liver disease, chronic lung disease, cystic fibrosis, dementia, diabetes mellitus, disability, heart conditions, HIV, mental health conditions, obesity (body mass index > 30), physical inactivity, pregnancy, primary immunodeficiencies, respiratory tuberculosis, smoking, and solid organ or stem cell transplant, and use of select immunosuppressive medications. Pregnancy was captured in the 301 days preceding the index date. Other underlying medical conditions were captured in the 12-month preindex period. Code sets and medication lists used to identify underlying medical conditions were previously reported in Kopel et al [2]. Patients with at least 1 of these medical conditions were considered “high risk” for severe COVID-19 outcomes.

Data Analysis

Propensity scores predicting receipt of mRNA-1273.815 were calculated for each subject using a multivariable logistic model adjusted for all covariates outlined above. Stabilized and truncated weights were used to reweight the study sample using the IPTW methodology. Sample balance before and after IPTW was assessed by calculation of the standardized mean differences (SMD). SMDs with absolute values >0.1 indicated covariate imbalance. Covariates were reported descriptively before and after weighting with means and standard deviations for continuous variables and N and percent for categorical variables.

Incidence rates, Kaplan-Meier curves, and unadjusted HRs were reported for the unweighted sample, and adjusted HRs were reported for the weighted sample. Unadjusted HRs were calculated using a Cox regression model with mRNA-1273.815 vaccination as the only predictor. Adjusted HRs were calculated for the weighted sample using a weighted Cox regression model that included mRNA-1273.815 vaccination status and any covariates with an SMD greater than 0.1 after IPTW. Subsequently, the unadjusted and adjusted VE for each outcome were calculated as 100 × (1 – HR) and reported with 95% CIs.

This analysis was conducted using SAS V9.4.

RESULTS

This study included 1 718 670 individuals; 859 335 adults who received the mRNA-1273.815 vaccine between 12 September 2023 and 15 December 2023 (exposed cohort), met all the study inclusion criteria, and were directly matched 1:1 to an unexposed adult (Figure 2). Of these, most had documentation of previous vaccination and some had documentation of previous infection; notably, approximately 70% (N = 605 816) of each cohort had previously received the COVID-19 bivalent vaccine and >85% had a record of previous COVID-19 vaccination.

The exposed and unexposed cohorts were well balanced after weighting, with no SMDs greater than 0.1. After weighting, the mean (standard deviation) age in both cohorts was 63 (16) years; 57% were female, 60% were White, 6% were Black, and more than 80% received their most recent COVID-19 vaccine more than 180 days before the index date (Table 1). Approximately 80% (N = 686 135) of the study population was at least 50 years old and 54% (N = 465 061) were at least 65 years old (Supplementary Table 4).

Preweighting, the majority of individuals, including 61.5% (N = 528 465) of the mRNA-1273.815 cohort and 66.4% (N = 570 783) of the unexposed cohort, had at least 1 underlying medical condition associated with an increased risk of severe COVID-19 outcomes (Supplementary Table 4). The most common underlying medical conditions preweighting were obesity (exposed: 22.5% and unexposed: 25.1%), diabetes mellitus (20.2% and 23.8%, respectively), and heart conditions (14.5% and 17.6%, respectively) (Table 1).

The overall median (interquartile range) follow-up time was 63 (44–78) days. The median follow-up time for the age ≥50 years, age ≥ 65 years, and high-risk cohorts were 64, 65, and 63 days, respectively (Supplementary Table 4). For the overall population, 794 COVID-19–related hospitalizations and 8710 COVID-19–related medical encounters were identified preweighing. Of these, 88.2% of hospitalizations and 67.4% of COVID-19–related medical encounters occurred among adults aged at least 65 years. For reference, counts of COVID-19–related outpatient encounters and positive laboratory tests for COVID-19 are reported in Supplementary Table 4.

The incidence rate of COVID-19 hospitalizations per 100 000 per week was 7.38 (95% CI, 6.80–7.99) in the unexposed cohort and 2.32 (95% CI, 2.01–2.68) in the exposed cohort throughout the entire follow-up period. The incidence rate of any medically attended COVID-19 per 100 000 per week was 65.18 (95% CI, 63.44–66.95) in the unexposed cohort and 41.44 (95% CI, 40.05–42.85) in the exposed cohort. Kaplan-Meier curves of outcomes show that the incidence rate was fairly steady through the follow-up period (Figure 3).

The adjusted VE (95% CI) against COVID-19–related hospitalizations was 60.2% (53.4–66.0) among adults at least 18 years old (Figure 4). Similarly, the VE against COVID-19–related hospitalizations was 61.1% (54.3–66.9) among adults ≥50 years of age, 60.5% (53.3–66.6) among adults ≥65 years of age, and 58.7% (51.3–65.0) among adults with underlying medical conditions.

The adjusted VE against any medically attended COVID-19 was 33.1% (30.2–35.9) among adults at least 18 years old. VE against medically attended COVID-19 was 35.3% (32.2–38.2) for adults ≥50 years old, 38.7% (35.4–41.9) among adults ≥65 years old, and 34.5% (31.2–37.6) among adults with at least 1 underlying medical condition. Unweighted, unadjusted VE estimates are presented in Supplementary Table 5.

DISCUSSION

In this analysis of 1 718 670 US adults aged ≥18 years, vaccination with mRNA-1273.815, an Omicron XBB.1.5-containing COVID-19 vaccine, was associated with significant protection against COVID-19–related hospitalizations and medically attended COVID-19. Notably, the significant effectiveness of mRNA-1273.815 was observed in all subgroups, including older adults and those with specific underlying medical conditions that may increase their risk of severe COVID-19. These findings support CDC recommendations to remain up to date with COVID-19 vaccination and underscore the benefit of receiving mRNA1273.815 for the general population and those at higher risk for COVID-19–related morbidity and mortality.

These findings are consistent with other real-world effectiveness studies of 2023–2024 Omicron XBB.1.5-containing COVID-19 vaccines. Initial effectiveness studies conducted in Denmark and the Netherlands, primarily with the BNT162b2 XBB.1.5-adapted vaccine, reported effectiveness estimates of greater than 70% against COVID-19–related hospitalizations over a short follow-up period [11, 12]. Real-world studies conducted in the United States estimated the effectiveness of the 2023–2024 COVID-19 vaccines to be between ∼40% and 60% for COVID-19 outcomes, including symptomatic infection, illness resulting in an emergency department/urgent care visit, or hospitalization with a median follow-up period of 30–50 days [10, 13, 14, 20]. Although these studies differ in methods, population, outcomes, and duration, they are consistent with this study's findings and demonstrate the significant benefit of vaccination with the updated 2023–2024 COVID-19 vaccines. Furthermore, the follow-up postvaccination in this current study is one of the longest, exceeding 60 days. As the circulating variants continue to evolve, additional analyses are needed to assess the duration of the effectiveness over time.

The results of this study are also consistent with neutralizing antibody data generated among individuals who received mRNA-1273.815. A study in which 50 individuals received mRNA-1273.815 demonstrated a 17.5-fold increase in neutralizing antibody levels against XBB.1.5 at day 29 postvaccination relative to prevaccination antibody titers [9]. A similar increase was observed against other SARS-CoV-2 variants, including EG.5.1, XBB.1.16, BA.4/BA.5, EG.5.1, BA.2.86, and JN.1, supporting significant cross-reactive immune responses from 2023–2024 Omicron XBB.1.5-containing vaccination across related SARS-CoV-2 variants. Data from a US CDC IVY study assessing the effectiveness of the updated 2023–2024 COVID-19 vaccines reported that, among the >900 samples that were sequenced from SARS-CoV-2–positive specimens in the study, only 16% had an XBB.1.5-like spike protein, whereas almost 80% had spike proteins of other XBB lineage variants and ∼5% had JN.1-like spike proteins [13]. These data suggest that the Omicron XBB.1.5-containing vaccine formulation offers protection from a range of XBB variants, including newer variants in the XBB lineage, like EG.5 and HK.3.

Four years since the start of the COVID-19 pandemic, a significant portion of the adult population in the United States has developed immune responses against the SARS-CoV-2 virus, whether through previous infections, vaccinations, or a combination of both [21]. This widespread seropositivity offers a degree of immunity within the community, which tends to diminish in the face of new, evolving virus variants. Although the current study included individuals regardless of their vaccination and infection history, the majority of the population in both cohorts received the bivalent vaccine and cohort adjustment included vaccination and infection history. Thus, the results of this study highlight the incremental protection provided by the additional dose targeting the circulating variants in a real-world setting regardless of exposure and/or vaccination history.

The majority of the US adult population has completed their primary series vaccination. However, as of May 2024, fewer than 1 in 4 adults aged ≥18 years and approximately 40% of adults aged ≥65 years had received a 2023–2024 updated COVID-19 vaccine [22]. Studies that include individuals who were previously vaccinated or infected and show the incremental protection provided by updated vaccines, especially against severe disease, are essential and should be routinely communicated to increase vaccine confidence among healthcare providers and the general population.

Limitations

Study limitations include a lack of generalizability to uninsured individuals or those with less stable insurance, as individuals were required to have continuous enrollment in claims. In addition, inherent to most noninterventional studies using real-world data, selection bias may be present due to the nonrandomized nature of data collection. To mitigate this, direct 1:1 matching and propensity score weighting were conducted using a broad range of covariates. After IPTW, balance was achieved on all measured characteristics. However, the possibility of residual unmeasured confounding persists because there may be characteristics not accounted for with the chosen study covariates, and some included covariates, such as time since last COVID-19 infection, might not be well captured in the data source if patients used an at-home test and never sought medical attention for their infection. Additionally, IPTW cannot account for differences in risk-taking behavior between the 2 cohorts. Uptake of the XBB.1.5-containing vaccines was lower than uptake of prior COVID-19 vaccines, and individuals who received an XBB.1.5-containing vaccine may be more aware of the risks of COVID-19 and more likely to take other steps to mitigate the risk of COVID-19 infection.

Furthermore, like all retrospective database studies, this study is subject to limitations because of data completeness. For example, the absence of a vaccination record does not guarantee that a patient did not receive a vaccine. Although we cannot determine the degree of missingness, this study leveraged a linked database enriched with open claims sources, closed claims sources, and EHR sources for the identification of exposures, covariates, and outcomes during a period of continuous claims enrollment, which reduces but does not eliminate the risk of missingness. Since the COVID-19 vaccine was commercialized during the 2023–2024 respiratory season and not distributed under an emergency use authorization, it was more likely that documentation of payment would be included in the claims dataset. Furthermore, by requiring documentation of age, sex, and state of residence and matching on criteria like prior COVID-19 vaccination, we reduced the risk of bias in missingness between the 2 cohorts. In addition, outcomes identification in the dataset may be limited in terms of specificity and sensitivity. For example, any medically attended COVID-19 could potentially include asymptomatic infections, which resulted in a positive COVID-19 test, but would exclude infections that resulted in a positive at-home COVID-19 infection for which the individual never sought medical care.

In this analysis, we were not able to differentiate between hospitalization “due to” versus hospitalization “with” COVID-19. The selected hospitalization outcome was based on a validation study of several US claims data sources in 2020, estimating the positive predictive value of the U07.1 diagnostic code alone and in any position to range from 94.1% to 81.2% for identifying COVID-19 hospitalizations [23]. This approach is consistent with prior publications evaluating the effectiveness of COVID-19 vaccines [18, 24]. Notably, the robustness of the dataset and completeness of results are further supported by the hospitalization incidence rates in this study (unexposed group), which were consistent with those reported by the CDC [3].

Last, several factors may contribute to an over- or underestimate of VE. For example, we used a 7-day window between vaccination and the cohort entry data, which may result in capturing some infections among vaccinated individuals before the development of a full immune response. This approach is consistent with other studies of the XBB.1.5 updated COVID-19 vaccines but may result in an underestimation of VE [11, 12]. On the other hand, although SARS-CoV-2 still does not have a clear seasonality, this analysis included only part of the winter season, which is usually characterized by an increase in incidence rates [25]. Partial season analyses, similar to the study presented here, aim to inform vaccine recommendations in a timely manner but are limited in scope to the time period when VE is likely highest. Vaccine effectiveness is anticipated to wane with increasing time between vaccination and virus exposure and with the evolution of the circulation of new variants. Thus, although this study provides a follow-up duration of more than 60 days, as SARS-CoV-2 continues to evolve, further analyses are needed to determine the durability of protection.

CONCLUSION

In this real-world effectiveness assessment, the mRNA-1273.815 vaccine (2023–2024 Omicron XBB.1.5-containing mRNA COVID-19 vaccine) provided significant protection against COVID-19-related hospitalizations and medically attended COVID-19. These findings are consistent with effectiveness estimates of 2023–2024 updated COVID-19 vaccines overall, demonstrating the incremental value over earlier vaccinations and the continued role of updated COVID-19 vaccinations in protection against COVID-19–related outcomes.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. Christopher Adams, an employee of Veradigm, provided programming support and quality assurance for this analysis. This support was funded via a contract with Moderna.

Author contributions. Conceptualization: J.A.M., H.K., A.B.A., D.M., M.A.M., T.L., and V.H.N. Methodology: all authors. Software: N.Z. Validation: A.B., J.P.W.-J., N.Z., and I.H.W. Formal analysis: A.B., J.P.W.-J., N.Z., and I.H.W. Investigation: A.B., J.P.W.-J., N.Z., and I.H.W. Resources: M.B. Data curation: N.Z. Writing—original draft preparation: J.P.W.-J., J.A.M., and H.K. Writing—review and editing: all authors. Visualization: J.P.W.-J., J.A.M., and H.K. Supervision: A.B., J.A.M., D.M., H.K., and M.B. Project administration: A.B. and M.B. Funding acquisition: J.A.M., D.M., H.K., and M.B. All authors have read and agreed to the published version of the manuscript.

Financial support. This work was supported by Moderna, Inc. via a contract with Veradigm.

Data availability statement. The data that support the findings of this study were used under license from Veradigm and Komodo Health. Because of data use agreements and their proprietary nature, restrictions apply regarding the availability of the data. Further information is available from the corresponding author.

Ethics approval and consent to participate. The linked dataset only contains deidentified data as per the deidentification standard defined in Section §164.514(a) of the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Privacy Rule. The process by which the data are deidentified is attested to through a formal determination by a qualified expert as defined in Section §164.514(b)(1) of the HIPAA Privacy Rule. Because this study used only deidentified patient records, it is therefore no longer subject to the HIPAA Privacy Rule and is therefore exempt from institutional review board approval and for obtaining informed consent according to US law. This study was conducted in compliance with the Declaration of Helsinki and used only deidentified data.

References

Author notes