Effectiveness of COVID-19 mRNA Vaccines in Preventing COVID-19-Associated Outpatient Visits and Hospitalizations Among American Indian and Alaska Native Persons, January–November 2021: A Test-Negative Case-Control Analysis Using Surveillance Data

Abstract

Health and social inequities affect the >9 million people living in the United States who self-identify as American Indian or Alaska Native (AI/AN), especially those living on tribal lands [1, 2]. Relative to members of other racial/ethnic groups, AI/AN persons experience higher rates of poverty, experience greater challenges accessing health care services, and are disproportionately affected by both noncommunicable and communicable diseases, including coronavirus disease 2019 (COVID-19), the disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [1–5]. When adjusted for age, AI/AN persons are 2.5 times more likely to be hospitalized and 2.1 times more likely to die due to COVID-19 than non-Hispanic White persons [6]. In addition to COVID-19's effects on individuals, entire AI/AN communities have experienced threats to kinship and well-being: AI/AN children have been shown to be 4.5 times more likely to experience caregiver loss compared with White children [7]. Because of discrepancies between self-reported race/ethnicity and that recorded on death certificates, these numbers likely underestimate the true COVID-19 burden for AI/AN individuals [8].

In December 2020, based on safety and efficacy data from phase III clinical trials [9–11], 2 messenger RNA (mRNA) COVID-19 vaccines were granted Emergency Use Authorization (EUA) by the Food and Drug Administration for use among persons aged ≥16 years (BNT162b2 vaccine, Pfizer-BioNTech) [12, 13] or ≥18 years (mRNA-1273 vaccine, Moderna) [14]; in February 2021, Janssen's adenovirus-vector COVID-19 vaccine (Ad.26.COV.S) was also approved under EUA for use in persons aged ≥18 years [15]. Observational studies conducted in the United States shortly after vaccine rollout reported vaccine effectiveness (VE) similar to efficacy demonstrated in trials [16–22]. However, opportunities for AI/AN communities to participate in COVID-19 vaccine clinical trials were limited: AI/AN individuals comprised only 1.3% and 0.8% of the Pfizer-BioNTech and Moderna US trial participant populations, respectively, despite comprising 2.9% of the US population [9, 23]. Representation of AI/AN individuals in observational studies of VE has also been poor.

Many tribes have responded to the pandemic with a sense of cultural and collective responsibility, which has translated to having among the highest vaccine coverage rates of any racial/ethnic group [24]. As of January 4, 2023, 64.7% of AI/AN persons age 5 years and older are fully vaccinated with a primary series, and 77.7% have received at least 1 dose of a COVID-19 vaccine [25]. AI/AN-specific VE estimates are urgently needed given the higher prevalence of risk factors for severe COVID-19 and the disproportionate burden faced by these communities. To fill this data gap, we conducted a multisite, case-control study among AI/AN individuals in 3 tribal populations to estimate the effectiveness of SARS-CoV-2 mRNA vaccines against COVID-19-associated outpatient visits and hospitalization in AI/AN populations.

METHODS

This study was reviewed and approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB) and the ethics review board for each participating tribe (Arizona sites: Navajo Nation Human Research Review Board and Phoenix Area Indian Health Service IRB; Alaska sites: Alaska Area IRB, Alaska Native Tribal Health Consortium [ANTHC], Southcentral Foundation [SCF], and Yukon Kuskokwim Health Corporation [YKHC]). The Centers for Disease Control and Prevention (CDC) IRB relied on the Johns Hopkins IRB [26]. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines [27].

Data Source

We used data from the Respiratory Surveillance in Native American Children and Adults Study, an active, health care facility-based surveillance system established in 2019 for symptomatic acute respiratory illness (ARI), expanded to include COVID-19 in 2020 (Supplementary Table 1), among AI/AN persons living on the Navajo Nation or White Mountain Apache (WMA) Tribal lands in Arizona and in the Anchorage municipality or in 58 tribal communities in the Yukon-Kuskokwim Delta (YKD) in Alaska. In Arizona, health care is provided to tribal members through Indian Health Service hospitals, tribal health care facilities, and private health care facilities. In Anchorage, medical care is provided for AI/AN persons at the tribal health care facility Alaska Native Medical Center (ANMC), which is jointly owned and operated by ANTHC and SCF. The tribally operated YKHC provides health care services to AI/AN persons in the YKD. Surveillance was conducted at 8 participating facilities, described in Supplementary Table 2. Surveillance in inpatient units was conducted 6 days a week at study sites in Arizona and Alaska and staff attempted to enroll all eligible inpatients, even those admitted on non-surveillance days; outpatient surveillance was conducted 2–4 days a week at Arizona sites only.

Study Design

We conducted a test-negative case-control analysis to estimate mRNA COVID-19 VE against laboratory-confirmed COVID-19-associated (1) outpatient visits and (2) hospitalizations (Table 1). The study population included AI/AN persons age-eligible for COVID-19 vaccination (Supplementary Table 3) presenting for medical care at a participating site between January 1, 2021, and November 30, 2021, and tested for SARS-CoV-2 via molecular testing (reverse transcription polymerase chain reaction [PCR] or nucleic acid amplification test [NAAT]) or antigen test. Those with a positive SARS-CoV-2 result within the 10 days before illness onset or 4 days following their visit/admission and ≥1 symptom consistent with COVID-19-like illness were cases (Table 1). Those with negative results and ≥1 symptom consistent with COVID-19-like illness were test-negative controls.

Data Collection

Following informed consent, information on participants’ sociodemographic and household characteristics, symptoms of COVID-19-like illness, underlying medical conditions, previous medical care, self-reported COVID-19 vaccination, risk factors for severe COVID-19 illness, and mask use was collected via interview. Electronic health records (EHRs) were reviewed to collect information regarding previous testing for SARS-CoV-2 (including date, test type, and results), COVID-19 vaccination, clinical course, and outcomes of the enrollment illness. Immunization Information Systems (IIS) were also reviewed to collect information on COVID-19 vaccination.

Midturbinate nasal swab research specimens were collected from every participant; if a specimen could not be collected, a salvaged clinical nasal specimen was used instead. Midturbinate swabs were placed in viral transport media, aliquoted, and stored at −80°C until testing. One aliquot was sent to Vanderbilt University Medical Center for testing via single-plex PCR for SARS-CoV-2. Testing for SARS-CoV-2 and interpretation of results were performed according to methods described in the CDC EUA protocol [28]. If the participant tested positive for SARS-CoV-2, the second aliquot was sent to Johns Hopkins University for viral genomic sequencing to determine SARS-CoV-2 lineage. Specimens were extracted using the Chemagic 360 system (Perkin Elmer) using methods and procedures previously described [29]. Clade and lineage were assigned if the genome's coverage was >70%.

Exposure, Outcome, and Covariate Assessment

The exposure of interest was COVID-19 vaccination status (complete primary series or unvaccinated) with an mRNA product at the time of illness onset (Table 1). Vaccination status was ascertained from an EHR or IIS (ie, “documented vaccination”) or from self-report with a known or approximate date of vaccination (ie, “patient self-report with date”). For the main analysis, “documented vaccination or patient self-report with date” was used to define vaccination status (Table 1). Discrepancies between EHR-documented and self-reported vaccination status were investigated thoroughly, including re-contacting participants if appropriate; we deferred to dates from EHRs/IIS unless the discrepancy was a result of a missing date in the EHRs/IIS, in which case we deferred to dates provided by the participant.

The primary outcomes were laboratory-confirmed COVID-19-associated outpatient visits (outpatient clinic and emergency department) and hospitalizations before and during Delta predominance (Table 1). Outpatient visits and hospital admissions were considered associated with COVID-19 if the SARS-CoV-2 result was positive from a specimen collected no more than 10 days before illness onset or 4 days after presentation for care. Participants may have had multiple clinical SARS-CoV-2 tests during their visit or hospital stay, in addition to research specimen testing; cases were defined if ≥1 test had a positive result, and controls were defined if all tests had a negative result. Because of difficulty accessing patient rooms and safety considerations during the pandemic, collection of research swabs was often delayed relative to participants’ presentation for care. Therefore, case status was maintained if the patient had a positive clinical molecular or antigen test but the VUMC PCR result was negative.

Statistical Analysis

Baseline characteristics were described by reporting frequency and proportion for categorical variables and median (Q1–Q3) for continuous variables. Missing data were reported where relevant. We compared the odds of complete primary vaccination with an mRNA vaccine with being unvaccinated among cases and controls, stratified by outpatient and inpatient status; due to small sample size, partially vaccinated persons were excluded from the VE analysis. We used unconditional, multivariable logistic regression to calculate an adjusted odds ratio (aOR) and estimated VE as (1 – aOR)*100%. We included the following variables in the adjusted models a priori: age, sex, month of illness, and enrollment site. No children aged 5–11 years had completed primary vaccination; therefore only persons age ≥12 years were included in analyses. Effect modification was evaluated by age, sex, community, time since vaccination, variant circulation, and mRNA product. We defined “pre-Delta” and “Delta predominance” according to community and state surveillance data [30, 31]. Delta predominance (>50% of sequenced specimens) began July 1, 2021, and lasted through the end of the analytic period (Supplementary Figure 1, Supplementary Table 4). We conducted sensitivity analyses to address potential misclassification and testing biases (eg, excluding participants without documented vaccination history, excluding participants whose case status was defined by antigen testing) (Supplementary Data).

All analyses were conducted using Stata 16 (StataCorp, College Station, TX, USA). Two-sided 95% CIs were calculated for each reported OR; 95% CIs that did not include 1 were considered statistically significant. P values were 2-sided, and statistical significance was set at P < .05.

RESULTS

Study Population

During the analytic period, 681 participants were enrolled (Figure 1). Characteristics of individuals who enrolled compared with those who declined or could not be enrolled are presented in Supplementary Table 5. After applying the definitions and exclusion criteria in Table 1, 244 inpatients (133 cases, 111 controls) and 230 outpatients (74 cases, 156 controls) were included for analysis (Figure 1). At the time of illness onset, 54 (73.0%) outpatient cases and 123 (78.8%) controls completed primary vaccination, with a median (interquartile range [IQR]) interval of 160.0 (104.0–220.0) days after receipt of the second dose (Table 2; Supplementary Table 6); thirty-seven (27.8%) hospitalized cases and 84 (75.7%) controls completed primary vaccination, with a median (IQR) interval of 155.0 (90.0–222.0) days. Of those receiving at least 1 dose of an mRNA vaccine, 233 (78.2%) received Pfizer-BioNTech and 89 (29.8%) received Moderna. Outpatient cases were more likely to have hypertension and/or heart conditions compared with outpatient controls; hospitalized cases were more likely to have obesity compared with hospitalized controls (Table 3). The proportion of participants with certain chronic medical conditions differed significantly by age group (Supplementary Table 7).

Figure 1.

Study inclusion flowchart. ^aPersons deemed eligible during screening process, but staff unable to approach or consent (eg, too ill to approach, transferred to outside facility before approached, mental status did not permit consent). ^bPersons deemed eligible during screening process, but upon approach determined ineligible (eg, not AI/AN, non-ARI illness). ^cExclusion categories were not mutually exclusive, and individual categories may not sum to total excluded. Abbreviations: AI/AN, American Indian and Alaska Native; ARI, acute respiratory illness; IRB, institutional review board; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; TN, test-negative; VE, vaccine effectiveness.

Open in new tabDownload slide

Among hospitalized participants, 10 (7.5%) cases and 12 (10.8%) controls were admitted to the intensive care unit (P = .38) (Table 2). The median length of hospital stay was shorter among vaccinated cases compared with unvaccinated cases (4 days vs 5; P = .01) (Supplementary Table 8). Six (4.5%) inpatient cases died during hospitalization (Table 2).

Vaccine Effectiveness

Among inpatients, the effectiveness of 2 doses of mRNA vaccine was 94.6% (95% CI, 88.0%–97.6%), adjusting for age, sex, enrollment location, and month of illness (Figure 2). Among outpatients, VE was 86.5% (95% CI, 63.0%–95.0%), adjusting for the same covariates. Within the strata of age, sex, enrollment location, and vaccine product, CIs for VE point estimates overlapped (Figure 2). VE point estimates were higher during the 14–149 days since vaccination (vs ≥150) and during the pre-Delta period (vs during Delta predominance; inpatient only), but CIs overlapped. Additional analyses by region, select comorbid conditions, and time since vaccination restricted to Delta predominance are presented in Supplementary Tables 9–11. Overall, results from sensitivity analyses were similar to the main analysis (Supplementary Table 12).

Figure 2.

Adjusted vaccine effectiveness of 2 doses of either mRNA product, overall and by select stratifications. ^aAdjusted for age, sex, site of enrollment, and month of illness onset. ^bTime since vaccination among participants completing primary vaccination ≥14 days before illness onset included a minimum of 21 days and a maximum of 312 days. ^cAdjusted for age, site of enrollment, and month of illness onset. ^dAdjusted for sex, site of enrollment, and month of illness onset; participants age 12–15 years not included in stratification because of small sample size (n = 10). ^eAdjusted for age, sex, and month of illness onset. ^fNo outpatients enrolled during the pre-Delta period. ^gNo outpatients age ≥65 years enrolled. ^hNo outpatients enrolled in Alaska. Abbreviation: No, number; VE, vaccine effectiveness.

Open in new tabDownload slide

DISCUSSION

This study provides evidence that mRNA COVID-19 vaccines were highly effective against laboratory-confirmed inpatient and outpatient COVID-19 among AI/AN persons before and during Delta predominance, underscoring the vaccines’ importance as tools to help reduce health disparities.

In our study, the effectiveness of 2 doses of mRNA COVID-19 vaccine against COVID-19-associated hospitalizations (94.6%) and outpatient visits (86.5) among AI/AN persons was high. VE was maintained among inpatients from pre-Delta (99.3%) to when Delta became predominant (91.7%); point estimates among outpatients were lower than inpatients during Delta predominance (80.8%), but confidence intervals overlapped. These findings are consistent with previous observational studies conducted before and during Delta predominance in the United States [16–18, 32]. In a test-negative design study among health care workers before Delta predominance, the effectiveness of 2 doses of mRNA COVID-19 vaccine against symptomatic illness in AI/AN persons was 93.7% [33]. It should be noted that this study defined cases based on test-positivity only, and it was conducted shortly after vaccine rollout; also, health care workers may differ from the general AI/AN population in terms of socioeconomic status and exposure risk, so comparisons with our results should be made cautiously. In a more recent test-negative design study conducted in Minnesota during Delta predominance, estimated VE among AI/AN persons was lower for COVID-19-associated hospitalization (Pfizer-BioNTech: 72%; Moderna: 76%) than our estimates of VE during Delta predominance [34]. When stratified by time since vaccination (using a similar time frame as our study, ≤26 vs >26 weeks), results were similar, but our estimates were still greater. Recent studies have indicated that risk of reinfection is lowest among vaccinated individuals previously infected before vaccination [35, 36], and cumulative rates of disease per 100 000 persons are lower in Minnesota compared with Alaska and Arizona, particularly in counties where AI/AN persons make up large proportions of the population [37]. The degree of protection from hybrid immunity likely depends on when the prior infection occurred; however, higher rates of disease among persons in our study may explain differences in reported VE. The limited number of studies with AI/AN-specific estimates underscores the need to improve racial/ethnic representation among scientific research participants.

Elders have borne a disproportionate burden of COVID-19-associated morbidity and mortality in AI/AN communities [4]. Traditional knowledge, histories, language, and values are irreparably lost with every tribal elder who has succumbed to COVID-19, and these losses are borne by entire communities [4]. Previous studies have reported reduced VE in older hospitalized adults [18, 34]. Although we also observed that VE point estimates decreased with age among inpatients, confidence intervals overlapped, and there were no significant differences in VE by age group. Notably, our VE point estimate was >80%, which suggests robust protection among older AI/AN adults and highlights the importance of effectively communicating vaccination as a tool to prevent severe disease in this vulnerable group.

VE point estimates were lower among those for whom it had been ≥150 days since vaccination (compared with 14–149 days), but confidence intervals overlapped. Previous studies have shown waning of VE for mRNA products, but to a lesser extent for severe outcomes (eg, hospitalization) compared with infection. One recent study demonstrated that during Delta predominance, VE declined from 92% in the 2 months after primary vaccination to 77% ≥5 months after vaccination among patients presenting at emergency departments and urgent care centers, and from 94% to 82% among inpatients [32]. Even when limited to the Delta-predominant period, VE was relatively maintained ≥150 days since vaccination among both outpatients and inpatients in our study. Small sample size may explain differences in declines in VE reported here compared with other studies.

This analysis is subject to limitations. Recruiting participants within the context of the pandemic made prospective enrollment challenging, and our sample size was low; we were unable to use more granular indicators of time (eg, week vs month of illness; shorter time frames since vaccination), and confidence intervals were wide among outpatients. Future analyses using the surveillance platform described herein will assess VE against Omicron variant sublineages, which has been shown to be reduced relative to other variants in recent publications [32]. Additionally, risk of outcome and/or exposure misclassification bias is inherent to case-control studies. Case misclassification—although still possible—was limited by using molecular test results for most participants. The use of EHR- and IIS-documented vaccination status likely mitigated risk of exposure misclassification. Furthermore, except for mask wearing, most participants did not answer questions regarding personal prevention strategies (eg, social distancing), which may have resulted in potential unmeasured confounding. All communities implemented strict lock-down and stay-at-home order policies during COVID-19 surges and mask mandates, with more than two-thirds of all cases and controls reporting always wearing a mask, so risk of exposure was unlikely to differ meaningfully between cases and controls. We did not have information on prior SARS-CoV-2 infection, which has been shown to modify VE estimates [38]. Finally, a large proportion of eligible patients declined enrollment or were unable to be consented. Individuals with more severe disease were more likely to decline participation, be excluded because of inability to consent, or be transferred to a higher level of care before staff were able to approach for enrollment. Staff revisited these patients and made every attempt to enroll whenever possible (and when deemed safe for patients and staff); however, this may have biased our characterization of SARS-CoV-2 illness towards less severe disease.

CONCLUSIONS

AI/AN persons have been disproportionately affected by COVID-19 disease and associated outcomes compared with other groups in the United States. Mitigating the disproportionate impacts of the COVID-19 pandemic for AI/AN persons has been possible through tribal leadership's implementation of aggressive public health policies, including rapid, community-informed deployment of COVID-19 vaccines. Steep declines in COVID-19 cases were observed after the introduction of vaccines in study communities [24, 39, 40]. The results from this study fill a critical data gap regarding COVID-19 VE in a disproportionately burdened group and underscore the importance of maintaining robust vaccination efforts to reduce health disparities. Continued efforts to ensure that community members are up to date with their COVID-19 vaccines are important.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Acknowledgments

The authors would like to thank the study participants and their families; Tribal and IHS partners; ANMC research team; Johns Hopkins Center for Indigenous Health research team; YKHC research team; Halasa Lab at VUMC; Mostafa Lab at JHU.

Financial support. This work was funded by the Centers for Disease Control and Prevention SHEPheRD Program (contract number 200-2016-91781). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention.

Author contributions. C.S.L., R.M.H., and L.L.H. contributed to funding acquisition. C.S.L., A.B., A.P.C., M.K., M.M., M.M.P., C.G.S., J.R.V., and L.L.H. contributed to study concept and design. C.S.L., R.M.H., D.E.V., A.B.B., R.M.C., and P.V. contributed to study administration. C.S.L., R.M.H., D.E.V., C.D., J.D., S.G., E.H., D.P., K.R., K.T., A.S.W., N.H., H.H.M., and R.J.S. contributed to data acquisition. N.H. and H.H.M. contributed to study resources. C.S.L., R.M.H., C.G.S., and L.L.H. contributed to data analysis, and C.S.L. and R.M.H. accessed and verified the underlying data. C.S.L., C.G.S., J.R., R.J.S., and L.H.H. contributed to the initial manuscript drafting. C.S.L., A.B., A.P.C., M.K., M.M., K.P., M.M.P., C.G.S., J.R.V., R.J.S., and L.H.H. contributed to data interpretation and to the discussion of the results. All authors contributed to final revisions of the manuscript. R.J.S. and L.H.H. contributed to study supervision. All authors had final responsibility for the decision to submit for publication.

Patient consent. We confirm that patients’ consent was obtained for inclusion in this study and that the design of this work was approved by local ethical committees. The names of authorizing bodies are included in the main text of the paper.

Data availability. Participant data collected on Tribal lands are owned by the participating Tribal Nations. Data can be made available upon request to the corresponding author (contact [email protected]) if consistent with the Institutional Review Board–approved protocol and if the disclosure is approved by the participating Tribes.

References

Author notes