Modeling Effectiveness of Testing Strategies to Prevent Coronavirus Disease 2019 (COVID-19) in Nursing Homes—United States, 2020 Free

Abstract

Background

Identifying asymptomatic individuals early through serial testing is recommended to control coronavirus disease 2019 (COVID-19) in nursing homes, both in response to an outbreak (“outbreak testing” of residents and healthcare personnel) and in facilities without outbreaks (“nonoutbreak testing” of healthcare personnel). The effectiveness of outbreak testing and isolation with or without nonoutbreak testing was evaluated.

Methods

Using published SARS-CoV-2 transmission parameters, the fraction of SARS-CoV-2 transmissions prevented through serial testing (weekly, every 3 days, or daily) and isolation of asymptomatic persons compared with symptom-based testing and isolation was evaluated through mathematical modeling using a Reed-Frost model to estimate the percentage of cases prevented (ie, “effectiveness”) through either outbreak testing alone or outbreak plus nonoutbreak testing. The potential effect of simultaneous decreases (by 10%) in the effectiveness of isolating infected individuals when instituting testing strategies was also evaluated.

Results

Modeling suggests that outbreak testing could prevent 54% (weekly testing with 48-hour test turnaround) to 92% (daily testing with immediate results and 50% relative sensitivity) of SARS-CoV-2 infections. Adding nonoutbreak testing could prevent up to an additional 8% of SARS-CoV-2 infections (depending on test frequency and turnaround time). However, added benefits of nonoutbreak testing were mostly negated if accompanied by decreases in infection control practice.

Conclusions

When combined with high-quality infection control practices, outbreak testing could be an effective approach to preventing COVID-19 in nursing homes, particularly if optimized through increased test frequency and use of tests with rapid turnaround.

Coronavirus disease 2019 (COVID-19) has posed a significant public health challenge for nursing homes in the United States [1, 2]. To prevent introduction and transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), it has been recommended that nursing homes implement precautions that include monitoring symptoms of residents and healthcare personnel (HCP), testing symptomatic persons promptly, and instituting infection control practices including, but not limited to, isolating individuals with COVID-19 [3]. In addition, because asymptomatic and presymptomatic infected individuals can transmit SARS-CoV-2, identifying such infected individuals early through serial testing is also recommended to control COVID-19 in nursing homes both as part of an outbreak response as well as in facilities not experiencing outbreaks [4–6].When responding to an outbreak, the Centers for Disease Control and Prevention (CDC) currently recommends testing all residents and HCP every 3–7 days until no new cases are identified. In addition, in facilities not experiencing an outbreak, serial testing of all HCP is currently recommended at intervals dependent on the level of county transmission [6].

Infection prevention and control strategies in nursing homes might be hampered by shortages of testing supplies [7] and the time-consuming effort needed to implement widespread testing strategies. The impact of various testing strategies for asymptomatic residents and HCP has not been well characterized; understanding the relative benefit of testing strategies may help prioritize resources. Our objective was to provide estimates of the effectiveness (ie, percentage of COVID-19 cases prevented) of either outbreak testing or combining outbreak and nonoutbreak testing for preventing COVID-19 in nursing homes under different scenarios of test frequency and performance. We also estimate the “efficiency” (number of tests needed to prevent a case) of both outbreak and nonoutbreak testing strategies.

METHODS

Definitions

“Outbreak testing” was defined as serial testing (eg, daily, every 3 days, weekly) of all residents and HCP immediately following recognition of an initial COVID-19 case. “Nonoutbreak testing” was defined as serial testing of HCP in the absence of a known COVID-19 case. Mathematical modeling was used to evaluate 2 paradigms for using testing to help control COVID-19 in nursing homes: outbreak testing that begins immediately following recognition of an initial case and nonoutbreak testing in facilities when outbreaks are not occurring combined with outbreak testing once cases are identified. Testing frequency, turnaround time, and sensitivity were varied to broadly reflect testing capabilities as of November 2020. The primary quantities of interest were the “effectiveness” of a testing strategy, defined as the percent of cases prevented, and the “efficiency” of a testing strategy, defined as the number of cases prevented divided by the number of tests used.

Evaluating these strategies for a “typical” nursing home required estimating 3 quantities through modeling: (1) the expected number of SARS-CoV-2 infections occurring in an outbreak without testing of asymptomatic persons, that is, infection control is guided only by testing and isolating symptomatic residents and HCP. This quantity is considered the “baseline” outbreak size because, conceptually, our evaluation is for paradigms of testing asymptomatic persons. (2) The expected number of SARS-CoV-2 infections occurring per outbreak when outbreak testing is conducted. The difference between (2) and (1) is the number of cases prevented by outbreak testing. (3) The expected percentage of SARS-CoV-2 outbreaks prevented by nonoutbreak testing.

Parameters used for the model and estimates were set as follows: during the early months of the pandemic the mean number of residents from Centers for Medicare & Medicaid Services (CMS)–certified nursing homes was 86 [8]. The HCP per resident ratio was set at 1.5:1, yielding 129 HCP at the facility based on the following rationale: analysis of the payroll-based journal data from the CMS suggest that, on average, 1.2 full-time equivalent HCP per resident are employed by each nursing home [9]. However, this likely represents a minimum estimate because some HCP work part-time. In some areas where facility-wide testing of HCP and residents has been conducted, the HCP per resident ratio has been as high as 2:1 (CDC, unpublished data). The ratio of 1.5 HCP per resident was chosen as closer to the midpoint of these estimates. Although these HCP estimates may include some who do not have direct resident contact, such HCP could still contribute to SARS-CoV-2 transmission (eg, HCP to HCP). Estimates of infectivity from asymptomatic or presymptomatic persons [10] varied over the course of illness and were used to calculate the expected reduction of transmission occurring from detecting these people through testing and isolating them (Supplementary Methods). All scenarios model introductions of SARS-CoV-2 into the facility through HCP only, for example, no visitors are permitted. A summary of model inputs can be found in Table 1.

Calculation of Estimates

To estimate the expected size of an outbreak at baseline, a Reed-Frost model in which the susceptible individuals in a population of size N have probability $p=RN−1$ of becoming infected at each stage of transmission [11] was used to model the transmission of SARS-CoV-2 during a nursing home outbreak. The Reed-Frost model was chosen because it is well suited for capturing stochasticity of transmissions in finite-sized congregate settings. The total population size is the number of residents plus the number of HCP. We estimated the reproductive number R₀ for outbreaks under the following conditions: the only testing occurring was of symptomatic persons, isolation of symptomatic persons occurred, and recommendations for other established infection control practices (eg, isolating patients, universal use of face masks and cloth face coverings) were in place. This estimate of R₀, based on data for nursing home outbreaks from Colorado during the early period of the pandemic (Supplementary Methods), is referred to as the “unmitigated” R₀ and used to generate the baseline outbreak size.

To determine outbreak size when using outbreak testing, modeling based on viral dynamics of SARS-CoV-2 was used to estimate the reduction in transmission μ expected from the testing strategy (Supplementary Methods). The mitigated reproductive number R₀’ was calculated by multiplying the unmitigated R₀ by the reduction in transmission μ expected from the testing strategy and used to estimate outbreak size using outbreak testing.

For nonoutbreak testing, the proportion of outbreaks prevented was set to equal μ. This corresponds to an assumption that an outbreak (at baseline) will typically be introduced by a single person so that the proportion of transmissions prevented equals the proportion of outbreaks averted and represents a best-case scenario for nonoutbreak testing regardless of the level of community transmission. As outlined in the Supplementary Methods, this was combined with estimates for outbreak testing to determine the effectiveness and efficiency of using both outbreak and nonoutbreak testing.

The “baseline” test sensitivity was modeled to vary during the course of illness with test sensitivity proportional to infectiousness early in the course of illness (Supplementary Methods). Peak sensitivity was set at 95% to reflect our best understanding of reported reverse-transcriptase polymerase chain reaction (RT-PCR) test characteristics [12]. To mirror realistic and recommended tests available, outbreak and nonoutbreak testing strategies were primarily evaluated with weekly and every-3-days testing intervals with either a 24-hour or 48-hour turnaround time and sensitivity corresponding to RT-PCR or an estimated sensitivity of 85% (compared with RT-PCR tests) with immediate turnaround time, that is, point-of-care test (Supplementary Methods). The lower sensitivity of 85% was chosen to match the lower end of the range of sensitivity reported for currently available point-of-care tests [13]. We also evaluated a hypothetical outbreak and nonoutbreak testing strategy in which a point-of-care test with 50% sensitivity is used for daily testing. These testing scenarios with lower sensitivity were evaluated because some investigators have advocated for the development and adoption of rapid tests with lower sensitivity [14]. We also evaluated weekly and every-3-days testing with a lower sensitivity (50%) point-of-care platform because of reports that existing point-of-care tests may have lower sensitivity for detecting asymptomatic individuals [15].

Sensitivity Analyses and Other Variations

We used the above analyses to evaluate the impact of adding nonoutbreak testing to outbreak testing with a 10% probability of SARS-CoV-2 introduction to a facility during the week that testing occurs, which corresponded to the overall national picture during July 2020–September 2020 (Supplementary Methods). As a sensitivity analysis, this probability was increased to evaluate the potential impact of nonoutbreak testing in areas with higher levels of community transmission.

We considered that implementing a more comprehensive testing strategy would require additional resources (eg, staff time, protective equipment). We postulated that these increases might have a deleterious effect on a facility’s ability to control transmission through adherence to infection control protocols. To estimate this effect, we calculated the impact of a relatively small (ie, 10%) decrease (from a default of 100%) in the effectiveness of preventing transmission by isolating infected persons when adding nonoutbreak testing to outbreak testing.

Additional sensitivity analyses used adaptations of 3 alternate infectivity profiles from the literature to calculate the above quantities (Supplementary Methods; Supplementary Table 1).

Administrative Information

Modeling calculations were conducted with the statistical software R (version 4.0.2, the R Foundation for Statistical Computing). Additional details about modeling can be found in the Supplementary Material. Code for the models can be found on the CDC Epidemic Prediction Initiative GitHub site [16].

This activity was reviewed by the CDC and was conducted consistent with applicable federal law and CDC policy (see eg, 45 C.F.R. part 46, 21 C.F.R. part 56; 42 U.S.C. §241(d); 5 U.S.C §552a; 44 U.S.C. §351 et seq.).

RESULTS

Effectiveness and Efficiency of Outbreak and Nonoutbreak Testing

At all currently recommended testing intervals with acceptable turnaround times (ie, 48 hours or less), outbreak testing alone is estimated to prevent 54% (weekly testing with 48-hour turnaround time) to 90% of cases (point-of-care testing every 3 days) associated with a COVID-19 outbreak (Table 2). Adding weekly or every-3-days nonoutbreak testing to outbreak testing prevented up to 8% of additional cases. Combining every-3-days outbreak and nonoutbreak testing with a point-of-care test prevented 95% of cases. In addition, at a given testing frequency and turnaround time, either increasing the frequency of the testing or decreasing the turnaround time for outbreak testing was estimated to prevent more infections than adding nonoutbreak testing with the same frequency and turnaround time. In an exploratory analysis, use of a daily point-of-care test with 50% sensitivity was estimated to prevent 92% of cases using outbreak testing alone and 97% of cases using both outbreak and nonoutbreak testing.

Combining outbreak and nonoutbreak testing in the setting of a 10% decrease in the effectiveness of isolating infected persons prevented fewer cases than outbreak testing alone with complete effectiveness of isolation for all testing frequencies, turnaround, and test sensitivity scenarios evaluated except when testing every 3 days with 24-hour turnaround time or point-of-care testing with 85% sensitivity. In these scenarios, adding nonoutbreak testing but decreasing effectiveness of isolating infected persons was estimated to prevent an additional 2%–3% of cases compared with outbreak testing alone with complete effectiveness of isolation.

The number of tests needed to prevent a case through outbreak testing alone (range, 41–145 tests/case prevented) was lower than through combining outbreak and nonoutbreak testing (range, 60–238).

Varying Likelihood of SARS-CoV-2 Introduction from the Community

When conducting outbreak and nonoutbreak testing every 3 days with a point-of-care test with 85% sensitivity, as the likelihood of SARS-CoV-2 introductions per week increased, the percent of cases prevented was estimated to remain unchanged at 95% (Figure 1). However, as the likelihood of SARS-CoV-2 introductions increased, the number of tests needed to prevent a case was estimated to decrease from 90 tests/case prevented at a 10% likelihood of introductions to 9 tests/case prevented at a 100% likelihood of introductions when combining outbreak and nonoutbreak testing (Figure 1).

Sensitivity Analysis

As a sensitivity analysis when using alternative infectivity profiles (Supplementary Table 1), the magnitude of benefit of outbreak testing varied (Supplementary Tables 2–4). For example, testing weekly with 48-hour turnaround prevented 31%–74% of infections depending on the infectivity profile. However, in all of these sensitivity analyses, adding nonoutbreak testing added a relatively small percentage benefit to outbreak testing alone, not exceeding 8% for any test turnaround/frequency/sensitivity regardless of infectivity profile. In addition, for all but 1 infectivity profile (Supplementary Table 4), for the majority of testing scenarios modeled, outbreak testing alone was superior to combining outbreak and nonoutbreak testing if combined testing was associated with decreases in effectiveness of isolating patients.

DISCUSSION

Preventing COVID-19 in nursing homes is a high priority for the pandemic response [17]. In our analysis, serially testing asymptomatic residents and HCP in response to an outbreak to guide infection control response prevented the majority of SARS-CoV-2 infections in a hypothetical nursing home with 86 residents and 129 staff. In addition, optimizing and prioritizing outbreak testing to facilitate increased test frequency with rapid turnaround times while maintaining high-quality infection control practice is estimated to be an efficient strategy while also nearly maximizing the number of infections prevented.

Ensuring robust symptom identification with prompt testing in response to a potential new SARS-CoV-2 infection and subsequently eliminating transmission from persons with COVID-19 through isolation are essential for responses using outbreak testing to be effective. Nursing homes with outbreaks frequently face staff shortages [18], which may limit capacity to perform necessary patient care activities, infection prevention activities, or increased testing. If redirecting resources from recommended infection control practices to testing leads to small reductions in the effectiveness of infection control practices, then the combination of outbreak and nonoutbreak testing could be less effective than just performing outbreak testing with high-quality infection control. These considerations are all the more important because real-life conditions in a nursing home might not correspond to ideal implementation. For example, isolation of patients can be difficult if there are no empty rooms; and isolated patients still need care and therefore could still pose some risk of transmission to staff.

These findings also support current recommendations from the CDC and CMS [6, 19] to prioritize testing of symptomatic residents and HCP first, followed by testing residents and staff in response to outbreaks, and finally nonoutbreak testing of asymptomatic HCP. The addition of nonoutbreak testing may provide a modest benefit, although it requires more testing resources to prevent each COVID-19 case. The testing resources required (per case prevented) did decrease as the likelihood of SARS-CoV-2 introductions (ie, level of community transmission) increased. The impact is still only modest, even when SARS-CoV-2 introductions are more frequent than observed in the United States in July 2020–September 2020. The reason that nonoutbreak testing only has a modest benefit is that the benefit of nonoutbreak testing relies on the ability to identify infected persons at key times in the course of their illness: the presymptomatic infectious period and, for asymptomatic individuals, the period of peak infectiousness. Both of these time periods are narrow (eg, presymptomatic period of 2–3 days) and easily missed without very frequent (eg, every 3 days or more often) testing [10, 14]. Furthermore, implementing high-quality outbreak response leads to smaller outbreaks that then reduces the added benefit of nonoutbreak testing because the outbreaks prevented through nonoutbreak testing are smaller in size.

The results described here also support current recommendations that performing outbreak testing every 3 days is the preferred frequency when initiating an outbreak response if this does not detract from other important activities such as infection control procedures [19]. In general, testing strategies with better turnaround time and higher frequency were estimated to be more effective if other conditions (eg, infection control practices) remained constant.

The federal government is providing nursing homes with point-of-care antigen testing platforms [20]. The analysis that we conducted suggests that the advantage of faster turnaround time from such platforms may outweigh the disadvantage of potential lower sensitivity. Practical considerations may still limit how frequently these tests can be performed through serial testing strategies. An exploratory analysis suggested that a hypothetical strategy using daily point-of-care testing with lower (eg, 50%) sensitivity could be as effective as, or even more effective than, relying on more sensitive tests used less frequently. Daily testing is likely not currently feasible in all nursing homes and would need widespread availability of inexpensive, simple testing options that could be run on easy-to-collect specimen types. In addition, making a test with 50% sensitivity available (that might facilitate daily testing) would require changes to the current regulatory framework [21–25].

These findings are subject to at least the following limitations. First, the analysis used mathematical modeling rather than directly studying an implementation. However, the model parameters have been reviewed as suitable to use for pandemic planning scenarios and are based on observational data about SARS-CoV-2 transmission [10]. The patterns seen and comparisons made from results, therefore, are likely reasonable even if the absolute estimates themselves may lack precision. Second, outbreaks in facilities have been assumed to be independent. Since some HCP work in more than 1 facility [4], the potential effectiveness of both testing strategies has been underestimated on a population level. Third, other unintended consequences of asymptomatic testing were not evaluated, such as identifying false-positives during nonoutbreak testing with population prevalence of COVID-19. Even if using a test with 99.4% specificity, most facilities without COVID-19 might expect at least 1 false-positive result after testing all of its staff. Fourth, there is uncertainty around the parameters used for the model. For example, the actual infectivity profile of SARS-CoV-2 infection is not known with certainty, although our main conclusions appear to be robust when evaluating several published infectivity profiles. As another example, the proportion of patients with asymptomatic infections has varied in the reported literature [26, 27]. Fifth, we have not accounted for how the accumulation of persons with immunity over the course of time might affect strategies. Finally, if infection control practices, in general, are not optimal, then the benefits of testing strategies will be less than what we have estimated. For example, we assume that outbreak testing occurs immediately after identification of a recognized case, while in practice delays could occur.

In summary, testing asymptomatic persons in response to an outbreak is an effective and efficient strategy to supplement optimized infection control practices to limit SARS-CoV-2 transmissions in nursing homes. The benefit of outbreak testing strategies may increase with more frequent outbreak testing with rapid turnaround times but, most importantly, require careful effort to ensure that infection control measures are effectively implemented.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. The authors thank Christopher Czaja, MD, DrPH (Colorado Department of Public Health and Environment) for discussions about nursing home outbreak response in Colorado early in the coronavirus disease 2019 pandemic.

Financial Support. This work was funded by the Centers for Disease Control and Prevention (CDC).

Disclaimer. The findings and conclusions in this report are those of the authors and do not necessarily represent the views and the official position of the Centers for Disease Control and Prevention.

Conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References